README.md
1# Sound Trigger Middleware
2
3## Overview
4Sound Trigger Middleware is a system service that exposes sound trigger functionality (low-power
5detection of acoustic events) to applications and higher-level system service.
6
7It has the following roles:
8- Isolating the soundtrigger HAL from potentially untrusted clients.
9- Enforcing correct behavior of the clients.
10- Enforcing correct behavior of the HAL and attempting to recover from failures.
11- Enforcing permissions for using soundtrigger functionality.
12- Serializing access to the HAL.
13- Logging soundtrigger usage in a comprehensive and consistent manner.
14- Generating a dumpsys report including current state and history of operations.
15- Providing a standard interface regardless which version of the HAL is implemented and gracefully
16 degrading operation whenever necessary.
17
18## Structure
19
20The service implementation can be divided into three main layers:
21
22- The "bottom layer" is concerned with HAL compatibility - making all HAL versions look and behave
23 the same.
24- The "middle layer" is concerned with the business logic of the service.
25- The "top layer" is concerned with exposing this functionality as a System Service and integrating
26 with other parts of the system.
27
28### HAL Compatibility Layer
29
30This layer implements the `ISoundTriggerHal` interface, which is the version-agnostic representation
31of the sound trigger HAL driver. It has two main implementations, `SoundTriggerHw2Compat` and
32`SoundTriggerHw3Compat` responsible for adapting to V2.x and V3 HAL drivers, respectively, including
33supporting their respective minor-version differences.
34
35This layer also includes several `ISoundTriggerHal` decorators, such as `SoundTriggerHalWatchdog`
36that enforces deadlines on calls into the HAL, and `SoundTriggerHalEnforcer` which enforces that
37the HAL respects the expected protocol.
38
39The decorator-based design is an effective tool for separation of aspects and modularity, thus
40keeping classes relatively small and focused on one concern. It is also very effective for
41testability by following dependency injection principles.
42
43### Business Logic Layer
44
45This layer also uses a decorator-based design for separation of concerns. The main interface being
46decorated is `ISoundTriggerMiddlwareInternal`, which closely follows the external-facing AIDL
47interface, `ISoundTriggerMiddlewareService`.
48
49Each of the decorators serves a focused purpose: for example, `SoundTriggerMiddlwarePermission`
50deals with enforcing permissions required for the various methods, `SoundTriggerMiddlewareLogging`
51logs all API usage, `SoundTriggerMiddlewareValidation` enforces correct usage of the protocol and
52isolates client errors from internal server errors.
53
54At the bottom of this decorator stack is `SoundTriggerMiddlewareImpl` / `SoundTriggerModule`, which
55are the adapter between `ISoundTriggerHal` and `ISoundTriggerMiddlwareInternal`, introducing the
56notion of having separate client sessions sharing the same HAL.
57
58### Service Layer
59
60This layer ties everything together. It instantiates the actual system service and the decorator
61stack. It also provides concrete connections to the Audio service (for negotiating sessions shared
62between Audio and Sound Trigger and for notifications about audio recording) and to the various HAL
63factories.
64
65This is the only layer that makes strong assumptions about the environment instead of relying on
66abstractions.
67
68## Error Handling and Exception Conventions
69
70We follow conventions for usage of exceptions in the service, in order to correctly and consistently
71distinguish the following cases:
72
731. The client has done something wrong.
742. The service implementation has done something wrong.
753. The HAL has done something wrong.
764. Nobody has done anything wrong, but runtime conditions prevent an operation from being fulfilled
77 as intended.
78
79The `SoundTriggerMiddlewarePermission` class would reject any calls from unauthorized clients,
80responding with the appropriate exception.
81
82The `SoundTriggerMiddlewareValidation` class does much of this separation. By validating the
83client's data and state, it would throw a relevant `RuntimeException` exception to the client
84without passing the requests down to the lower layers. Once that is done, any exception thrown from
85the underlying implementation can be assumed to be not the client's fault. If caught, they will be
86classified according to the following rule:
87
88- If they are `RecoverableException`s, they represent category #4 above, and will be presented to
89 the client as `ServiceSpecificException`s with the same error code.
90- Otherwise, they are considered an internal error (including HAL malfunction) and will be
91 presented to the client as `ServiceSpecificException(Status.INTERNAL_ERROR)`.
92
93Internally, we would throw `RecoverableException` whenever appropriate. Whenever a HAL malfunctions,
94`SoundTriggerHalEnforcer` is responsible for rebooting it and throwing an exception. A HAL death is
95considered a valid failure mode, and thus result in `RecoverableException(Status.DEAD_OBJECT)`,
96which ends up as a `ServiceSpecificException(Status.DEAD_OBJECT)` on the client side.
97
98## Notes About Thread Synchronization
99This component has some tricky thread synchronization considerations due to its layered design and
100due to the fact that it is involved in both in-bound and out-bound calls from / to
101external components.
102
103The following mutexes need to be considered:
104- Typically, a one or more mutexes that exist in every layer of the sound trigger middleware stack
105 to serialize access to its internal state or to external components.
106- Audio Policy Service lock. This one is external - it should be assumed to be held whenever we're
107 inside the `ExternalCaptureStateTracker.setCaptureState()` call stack *AND* to be acquired from
108 within our calls into `AudioSessionProvider.acquireSession()` /
109 `AudioSessionProvider.releaseSession()`.
110
111To avoid potential deadlocks, a strict locking order must be ensured whenever nesting locks. The
112order is:
113- Upper layers of the stack, starting from the top (i.e. may not attempt to acquire a higher-layer
114 mutex while a lower-layer mutex is being held) until `ISoundTriggerHw2`.
115- Audio Policy Service lock.
116- Lower layers of the stack, starting from `ISoundTriggerHw2` all the way down to the HAL.
117
118In order to enforce this order, some conventions are established around when it is safe for a module
119to call another module, while having its local mutex(es) held:
120- Most calls (see exceptions below) originating from SoundTriggerMiddlewareService simply propagate
121 down the decorator stack. It is legal to call into the next layer down while holding a local
122 mutex. It is illegal to invoke a callback with a local mutex held.
123- Callbacks propagate from the lower layers up to the upper layers. It is legal to hold a local
124 mutex within a callback, but **not** while call to an upper layer.
125- In order to be able to synchronize, despite the asynchronous nature of callbacks,
126 `stopRecognition()` and `unloadModel()` work differently. They guarantee that once they return,
127 the callbacks associated with them will no longer be called. This implies that they have to block
128 until any pending callbacks are done processing and since these callbacks are potentially holding
129 locks of higher-order mutexes, we must not be holding a local mutex while calling down. The proper
130 sequence for these calls is:
131 - Obtain the local lock if needed. Update/check local state as necessary.
132 - Call the respective method of the delegate ("downwards"). Once it returns, not more callbacks
133 related to this operation will be called.
134 - Obtain the local lock if needed. Update local state as necessary. Assume that state might have
135 changed while the lock has been released.
136 - Release the local lock.
137 - Invoke any synchronous callbacks if needed.
138- Calling from `SoundTriggerMiddlewareImpl` / `SoundTriggerModule` into the audio policy service via
139 `acquireSession()` / `releaseSession()` while holding the local lock is legal.
140- `setCaptureState()` calls, originating from Audio Policy Service, into the lower layers of the
141 stack may call into the HAL (specifically, they must invoke `stopRecognition()`, but must not
142 block on callbacks. For this reason, `SoundTriggerHw2ConcurrentCaptureHandler`, which is the
143 recipient of these calls, features a buffer and an additional thread, which allows the actual
144 stopping to be synchronous, as required, without having to block the call upon higher layers
145 processing the callbacks.
146