Skip to content

iOS: notify the ADM observer when a rollback drops an engine it saw created (m150) - #306

Closed
hiroshihorie wants to merge 1 commit into
hiroshi/adm-enable-rollback-compensation-m150from
hiroshi/adm-release-rolled-back-engine-m150
Closed

hiroshihorie wants to merge 1 commit into
hiroshi/adm-enable-rollback-compensation-m150from
hiroshi/adm-release-rolled-back-engine-m150

Conversation

@hiroshihorie

Copy link
Copy Markdown
Member

Stacked on #305. Review that first, the base moves to m150_release once it merges.

Problem

Found while testing #305 on device. The create step's rollback only clears the engine pointer. When a later step of the same operation fails, an observer that got OnEngineDidCreate for that engine never gets OnEngineWillRelease, so per-engine state it set up in did-create stays on an engine the device has dropped.

The Swift SDK MixerEngineObserver attaches its nodes in did-create and detaches them only in will-release. If the dropped engine is already gone when the operation is retried, the nodes come free and nothing visible happens. If anything keeps it alive, the retry aborts:

*** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio',
    reason: 'required condition is false: nil == owningEngine || GetEngine() == owningEngine'
  -[AVAudioEngine attachNode:]
  MixerEngineObserver.engineDidCreate(_:)
  AudioDeviceObserver::OnEngineDidCreate(AVAudioEngine*)
  webrtc::AudioEngineDevice::ApplyDeviceEngineState(...)
  webrtc::AudioEngineDevice::InitAndStartRecording(...)

Change

  • After a successful OnEngineDidCreate, push a rollback action that calls OnEngineWillRelease with the same engine. Covers both the device and manual rendering paths.
  • It runs before the create rollback, so the engine is still valid, and after the will-enable compensation from iOS: notify the ADM observer when an enable operation rolls back after will-enable (m150) #305, so a failed enable now looks like a normal cycle to the observer: did-create, will-enable, did-disable, will-release.
  • A did-create that returns an error gets no will-release, matching the did-disable contract from iOS: notify the ADM observer when an enable operation rolls back after will-enable (m150) #305. The result is only logged since the rollback cannot be aborted.
  • The will-release contract is documented in api/audio/audio_device.h and RTCAudioDeviceModule.h.
  • StopDeviceEngineAudioUnits() is left out on purpose. The release step needs it for an engine that ran, but in a rollback the start rollback has already stopped the engine, and reading outputNode on a fresh engine would instantiate the output unit as a side effect.

Testing

  • clang -fsyntax-only of audio_engine_device.mm (macOS and iOS) and RTCAudioDeviceModule.mm (iOS): no errors.
  • On device (iPhone 17 Pro Max, iOS 27) with Swift SDK main, an engine observer chain of a recorder plus the SDK mixer, the category check forced to fail right after will-enable, then a retry with .playAndRecord:

The create step pushes a rollback that only clears the engine pointer.
When a later step of the same operation failed, an observer that got
OnEngineDidCreate for that engine never got OnEngineWillRelease, so
per-engine state it set up in did-create stayed on an engine the device
had dropped. An observer that attaches its own nodes in did-create
leaves them attached there, and if anything keeps that engine alive the
next engine's attach of the same nodes aborts with an AVAudioEngine
"owningEngine" exception.

Push a rollback action right after a successful OnEngineDidCreate that
fires OnEngineWillRelease with the same engine. It runs before the
create rollback, so the engine is still valid, and after any will-enable
compensation, so the observer sees did-create, will-enable, did-disable,
will-release like a normal cycle. A did-create that returns an error
gets no will-release. The result is only logged. Applies to both the
device and manual rendering paths, and the will-release contract is
documented in both headers.
@hiroshihorie

Copy link
Copy Markdown
Member Author

Superseded by #312, which carries this commit unchanged (9dc7a01) on top of the #305 commits, plus a rule for teardown observer errors. The fault matrix and the rolled back create tests pass there on device. Closing in favor of #312.

hiroshihorie added a commit that referenced this pull request Nov 17, 2026
Supersedes #305, #306 and #309 (closed). #310 is a separate change.

## Problem

`ModifyEngineState` applies an engine transition step by step, rolls
back on any error, and commits the new state only when every step
succeeds. The engine observer hooks did not fit that model in two ways.

- Setup hooks could acquire without a release. When an enable failed
after a successful `OnEngineWillEnable`, or a create rolled back after a
successful `OnEngineDidCreate`, the observer was never told, so whatever
it acquired stayed held.
- Teardown hooks could roll back what cannot be undone.
`OnEngineDidStop`, `OnEngineDidDisable` and `OnEngineWillRelease` run
after `[engine stop]`, and the stop step registers no undo. An error
from them rolled the transition back with the engine already stopped, so
the committed state still said running. A later start saw no change and
never ran the engine, and an interruption end never restarted it.

## Change

The contract, documented in `api/audio/audio_device.h` and
`RTCAudioDeviceModule.h`:

1. Setup hooks (`OnEngineDidCreate`, `OnEngineWillEnable`,
`OnEngineWillStart` and the connect hooks) can fail the operation. A
non-zero result rolls it back and reaches the caller.
2. A setup hook that succeeded gets its release when the operation rolls
back. `OnEngineDidDisable` follows `OnEngineWillEnable`, and
`OnEngineWillRelease` follows `OnEngineDidCreate`.
3. Teardown hooks (`OnEngineDidStop`, `OnEngineDidDisable` and
`OnEngineWillRelease`) cannot fail the operation. A non-zero result is
logged and the operation completes.

One commit per rule:

- `68ac34a269` and `f92becad78`: will-enable compensation, port of #270
with its review follow-ups (previously #305).
- `9dc7a011fa`: did-create compensation (previously #306).
- `f300557860`: teardown errors are only logged, in the device and
manual rendering paths. Covers the interruption begin case of #309.

## Behavior change

A teardown hook error no longer reaches the caller. The Swift SDK's
`observerFail` audio test expects `stopLocalRecording()` to throw on a
failing `engineDidStop` and needs a matching update. React Native apps
whose JS did-stop handler throws get a log line instead of an error.

## Notes for review

- Not covered yet: an engine start failure after a successful
`OnEngineWillStart` still gets no `OnEngineDidStop`. The fault matrix
cannot inject a start failure, and current observers hold nothing from
will-start.
- #310 (double start at interruption end) and the interruption events
are separate changes and merge cleanly with this.

## Testing

- `clang -fsyntax-only` of `audio_engine_device.mm` and
`RTCAudioDeviceModule.mm` for macOS and iOS: no errors. The one iOS
warning is a pre-existing deprecation in `api/audio_options.h`.
- Fault injection matrix on device (iPhone 17 Pro Max, iOS 27),
apple-audio-lab `RollbackFaultMatrixTests`. A probe at the head of the
Swift SDK observer chain fails one of eight hooks once during one of
eight engine operations (capture start and stop, engine availability off
and on, interruption begin and end, mute and unmute in restart mute
mode). Each case then checks that bringing capture up runs the engine,
that taking it down stops it, and that every successful setup hook got
its release back at idle.
- Swift SDK 2.15.3 with the released LiveKitWebRTC 144.7559.11: 22 of 26
cases violate. 12 are setup failures that leak an acquire, 10 are
teardown failures that leave the engine stopped while the state says
running.
  - #306, which includes #305: 10 violations, all on the teardown path.
- Both numbers above predate the probe change below, which only affects
the three `willRelease` rows.
- A prototype of the teardown rule on top of #306 cleared the teardown
violations, except three that came from the probe itself (it failed
`willRelease` without forwarding it, which the probe now does).
  - This head: 0 of 26 cases violate.
- Other lab suites on this head, same device:
- Rolled back enable tests pass, including the did-disable order and the
release of the engine created for the failed enable.
- The #309 repro passes. A did-stop that fails at an interruption begin
no longer drops the interruption, and the end restarts the engine.
- A did-stop that fails on an app requested stop no longer throws, and
the next start runs the engine.
  - Interruption path and processing seed tests pass.
- The double start repros still hit `Check failed:
configuration_observer_ == nullptr` on this head, as expected without
#310.
hiroshihorie added a commit that referenced this pull request Nov 17, 2026
## Problem

Nothing keeps the engine from starting during an interruption. Any
transition from not running, a restart, or a recreate starts it while
`is_interrupted` is set, for example a capture start, or an engine
availability change while RTCAudioSession still considers the session
interrupted. The interruption end then runs the start step again through
`DidEndInterruption()`, although nothing stopped the engine:

- `OnEngineWillStart` fires a second time.
- A second `AVAudioEngineConfigurationChangeNotification` observer is
registered. Debug builds stop at `RTC_DCHECK(configuration_observer_ ==
nullptr)`. Release builds leak the first observer, so a later
configuration change runs `ReconfigureEngine` once per observer.

When RTCAudioSession's activation count is 0, which is always the case
under the Swift SDK, `updateAudioSessionAfterEvent` deactivates the
session at the interruption end. That stops the running engine without
the stop step, so the restart runs with the old observer still
registered.

## Change

- Start step: an interruption end starts the engine only when it is not
running.
- Stop step: at an interruption end, when the configuration observer is
still registered but the engine is not running, run the stop step first.
It removes the stale observer, and observers get `OnEngineDidStop`
before the restart.

## Notes for review

- Deactivating the session stops a running voice processing engine.
Measured on iOS 27: `setActive(false)` took 716 ms, `isRunning` was
false right after, and the engine posted one configuration change. That
is the activation count 0 path above.
- The normal path is unchanged. When the begin stopped the engine, the
observer is already removed and the engine is not running, so only the
start step runs at the end, as before.
- Independent of #305, #306 and #309, and merges cleanly with each. The
same code ships on `m144_release`, where React Native holds an
activation count above 0 and hits the double start.

## Testing

- `clang -fsyntax-only` of `audio_engine_device.mm` for macOS and iOS:
no errors.
- On device (iPhone 17 Pro Max, iOS 27) through apple-audio-lab. The
test posts a synthetic `AVAudioSession.interruptionNotification` began
while idle, starts local recording during the interruption, then posts a
synthetic ended with `shouldResume`.
- Activation count 1, raised through RTCAudioSession the way Flutter and
React Native hold it:
- Swift SDK 2.15.3 with the released LiveKitWebRTC 144.7559.11:
`OnEngineWillStart` fires twice with no stop in between.
- Local iOS debug build of `m150_release`: `Check failed:
configuration_observer_ == nullptr`.
- Local iOS debug build of this branch: `OnEngineWillStart` fires once
and the engine keeps running.
  - Activation count 0, the Swift SDK default:
- Swift SDK 2.15.3 with the released LiveKitWebRTC 144.7559.11: the
deactivation stops the engine, then `OnEngineWillStart` fires a second
time with the old observer still registered.
    - Local iOS debug build of `m150_release`: same check failure.
- Local iOS debug build of this branch: the ADM logs a stop, then a
start, and the engine runs.
- The other lab SDK tests (synthetic interruption stop and restart,
restart vs rebuild latency, processing seeds) pass on this build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant