Trillboards Sensing SDK — Android Integration Brief (agent-executable)

This is NOT the ad-monetization (IMA) guide. This is the camera + microphone + Wi-Fi + Bluetooth audience-SENSING SDK — the same agent-core engine that powers Trillboards' own retail screens. It produces audience Moments, scenes, demographics, and custom chips in the Trillboards Venue portal. Ad serving is a separate, optional SDK.

How to use this doc: paste it into Claude Code / Cursor inside your Android app's repo. The AI agent should follow it top-to-bottom and stop only at the Done criteria at the end. Everything needed is here.

What you're integrating

You are embedding the Trillboards Sensing SDK (com.trillboards:agent-core, from GitHub Packages) into an existing Android app that runs on in-store screens/boxes. After integration, each box's camera/mic/Wi-Fi/BLE feed on-device AI (face, age/gender, emotion, pose, audio, speech, BLE/Wi-Fi co-viewing), and the resulting audience intelligence attributes to that box's screen in the Trillboards Venue portal — automatically. No ads, no kiosk takeover; the SDK runs as a library inside the host app.

Prerequisites (you already have these)

  1. Your devices are already registered with Trillboards (one row per box) via authenticated POST /v1/partner/device, each with a stable external_id you chose (e.g. a serial like DD-H80F55). That external_id is the only per-box value the SDK needs at runtime. If a box is not yet registered, register it first (see Appendix A).
  2. Your Trillboards Partner API key (format trb_partner_… or tb_prt_…) is used only for provisioning actions such as registration. Do not bundle it in the device app for heartbeat, identity, or sensing startup.

The SDK passes your external_id to the backend, which already knows which screen+venue that box maps to (from registration). That is what makes the data attribute to the right location.

Step 1 — Add the dependency

In settings.gradle.kts (or root build.gradle), add the canonical public Trillboards Maven repository:


dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
        maven { url = uri("https://maven.trillboards.com") }
    }
}

The repository is anonymous-read. No GitHub token or Trillboards API key belongs in Gradle credentials.

In your app module build.gradle.kts:


dependencies {
    implementation("com.trillboards:agent-core:1.14.0")
    // Moonshine ASR runtime — enables on-device speech→structured output (FunctionGemma 270M).
    // agent-core ships ASR compileOnly; this supplies the runtime. Omit only if you don't want speech.
    implementation("com.trillboards.bundled:sherpa-onnx:1.12.26")
}

Build floor (required): minSdk = 26, Android Gradle Plugin 8.0+, JDK 17. The AAR is ~180 MB (bundled on-device models). If your app is below these, bump them.

Release candidates. A new version is first published to the staging repository, and it becomes the public latest only after that exact build passes a 48 h soak on an Ember-class tablet. To build a candidate before then, add the staging repository and pin the exact version:


maven { url = uri("https://maven.trillboards.com/staging") }
// app module: implementation("com.trillboards:agent-core:<candidate version>")

Staging has no latest, so a dynamic version (1.+) never resolves to a candidate. A staged version never changes, and the public release is the same bytes: what you tested as the candidate is what ships. The version directory's build-info.json lists the SHA-256 of the AAR and the POM. Remove the staging line once the version is public.

Step 2 — Permissions + foreground service

In AndroidManifest.xml, declare (inside <manifest>):


<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.BLUETOOTH_SCAN" android:usesPermissionFlags="neverForLocation" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.NEARBY_WIFI_DEVICES" android:usesPermissionFlags="neverForLocation" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_CAMERA" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />

Grant the runtime permissions (CAMERA, RECORD_AUDIO, ACCESS_FINE_LOCATION, BLUETOOTH_SCAN, and — on Android 13+ / API 33+ — NEARBY_WIFI_DEVICES) at app start (or via MDM auto-grant on managed fleets). BLUETOOTH_SCAN and NEARBY_WIFI_DEVICES are runtime-dangerous: declaring them (or letting the AAR merge them in) is not a grant, and skipping either request leaves that radio silently reporting nothing while the other looks healthy. WiFi scanning additionally needs ACCESS_FINE_LOCATION at every API level — on 33+ it is required alongside NEARBY_WIFI_DEVICES, because WifiManager.startScan()/getScanResults() are the carve-out NEARBY_WIFI_DEVICES does not cover — and the AAR caps its own declaration at maxSdkVersion="30", so declare an uncapped one yourself on API 31+. The per-API-level requestPermissions snippet is in the CTV Measurement SDK — Partner Integration Guide § Step 1b.

Foreground service: the SDK binds the camera to the app process lifecycle. If your screen app is always in the foreground (kiosk/always-on display), no extra work is needed. If you sense while the app is backgrounded, declare a foreground service with foregroundServiceType="camera|microphone" and call startForeground() before sensing — otherwise Android stops the camera in the background.

Step 3 — Start the SDK

Call this once at app start (e.g. Application.onCreate() or your main Activity), passing this box's external_id as deviceId:


import com.trillboards.sdk.SensingSdkConfig
import com.trillboards.sdk.TrillboardsSensingSdk

TrillboardsSensingSdk.start(
    context = applicationContext,
    config = SensingSdkConfig(
        deviceId = "DD-H80F55"   // ← THIS box's external_id (the one you registered)
        // speech/ASR + on-device FunctionGemma 270M run by default (sherpa-onnx dep added in Step 1)
    )
)

That's the whole integration. The SDK resolves this box's screen from deviceId (check-screen), handles any privileged sensing-channel credential internally, opens the camera/mic, runs on-device AI, and emits attributed observations. Heartbeat liveness and targeting-identity ingestion are unauthenticated after registration. To stop: TrillboardsSensingSdk.stop(applicationContext).

deviceId MUST equal the external_id you registered for this box. That's the entire attribution contract. If you templatize the app across a fleet, inject each box's external_id at provisioning time (build config, MDM app-config, or a per-device file).

Step 4 — Verify it worked (do this, don't skip)

  1. Run the app on a real device with a camera in view of people (or yourself).
  2. In logcat, confirm: TrillboardsSensingSdk: Resolved screen binding: screenId=… and FrameCaptureManager: ... SENDING TO API ... screenId=<non-null>.
  3. In the Trillboards Venue portal (/earner), open that box's screen → the Moments card populates within a few minutes (scenes, demographics, custom chips). That is success.

What you get (in the portal + via the partner API)

Step 5 — Proximity graph: co-install the Measurement SDK (BLE/Wi-Fi)

The Sensing SDK is camera + audio. Wi-Fi/Bluetooth proximity (the co-viewing graph) is produced by the CTV Measurement SDK, which runs the scans on its own scheduler — so proximity never weighs down the camera/audio path. To capture the full sensor suite, co-install it with the same deviceId so both attribute to the same screen:


// app/build.gradle.kts (same GitHub Packages repo as agent-core, from Step 1):
dependencies {
    implementation("com.trillboards:ctv-measurement:1.3.2")
}


import com.trillboards.measurement.MeasurementConfig
import com.trillboards.measurement.TrillboardsMeasurement

TrillboardsMeasurement.initialize(
    applicationContext,
    MeasurementConfig.Builder(deviceId = "DD-H80F55").build()
)
TrillboardsMeasurement.setConsentStatus(true)
TrillboardsMeasurement.startScheduledScans()

This SDK also reports the box's hardware (incl. RAM) and, when the platform exposes it, the real resettable GAID/AAID on its heartbeat. It never substitutes the registered device or screen ID for a MAID. The hardware report is what lets the backend serve the FunctionGemma 270M model to capable (≥ 2 GB) boxes. Camera-only? Skip this step. Full suite (proximity + on-device speech LLM)? Co-install it.

Both SDKs expose an ExtendedIdentityProvider for host-owned, exact targeting identities. Supply an already-computed SHA-256 email digest as hem_sha256 or a current opaque UID2 advertising token as uid2_token. HEM is lowercased once and never re-hashed; UID2 remains case-sensitive and should carry its expiry. These values stay separate from MAID and leave the backend as OpenRTB user.eids; MAID leaves as device.ifa.

Step 6 — Report your own plays, trips and taps (optional, agent-core 1.22+)

Three calls tell Trillboards what YOUR app owns, so the audience the SDK measured is attributed to it: the ads your app plays, your taxi meter, and what riders do with the screen. They do nothing unless you call them.


import com.trillboards.sdk.HostInteractionType
import com.trillboards.sdk.TrillboardsSensingSdk

// An ad your app played. creativeRef is the content hash POST /v1/partner/creatives returned.
TrillboardsSensingSdk.reportPlay(
    playId = "play-7f3a",                       // your unique id for this play
    creativeRef = "sha256:6c4b8f3e2a1d7b90c5e42f18a3d6b7e90124fa5c83b16d0e7f294a5c81b3d6e1",
    completedAtEpochMs = System.currentTimeMillis(),
    durationSeconds = 15.0,
    impressionId = null,                        // ONLY for an ad Trillboards served (see below)
)

// The meter: number the changes of one trip 0, 1, 2...
TrillboardsSensingSdk.setTripState(tripId = "trip-9", eventSeq = 0, meterOn = true, stateAtEpochMs = startedAt)
TrillboardsSensingSdk.setTripState(
    tripId = "trip-9", eventSeq = 1, meterOn = false, stateAtEpochMs = endedAt,
    fareCents = 2450, tipCents = 400, paymentType = "card", passengerCount = 2, destination = "JFK T4",
)

// What a rider did: TAP, QR_SHOWN, QR_SCANNED or DWELL_PROMPT.
TrillboardsSensingSdk.reportInteraction("tap-1", HostInteractionType.TAP, tappedAt, creativeId = "cr-9", tMs = 8000)

From Java they are static methods with short overloads: TrillboardsSensingSdk.reportPlay("play-7f3a", creativeRef, System.currentTimeMillis());, TrillboardsSensingSdk.setTripState("trip-9", 0L, true, startedAt);, TrillboardsSensingSdk.reportInteraction("tap-1", HostInteractionType.TAP, tappedAt);.

How they behave:

Step 7 — Seatbelt reminders (optional, agent-core 1.24+)

The SDK tells your app when the rear riders are unbuckled while the cab moves, so your app can show a reminder. Set a listener; nothing is delivered without one.


import com.trillboards.sdk.HostInteractionType
import com.trillboards.sdk.TrillboardsSensingSdk

TrillboardsSensingSdk.setSafetyEventListener { reminder ->
    showSeatbeltReminder(reminder.seatSide)            // your UI
    TrillboardsSensingSdk.reportInteraction(
        reminder.eventId, HostInteractionType.SEATBELT_REMINDER_SHOWN, System.currentTimeMillis(),
    )
}

From Java: TrillboardsSensingSdk.setSafetyEventListener(reminder -> { ... });. Pass null to remove the listener.

How it behaves:

On-device speech LLM (FunctionGemma 270M)

Performance notes (low-/mid-tier boxes, e.g. Snapdragon 695 / 3.5 GB)

Troubleshooting

Appendix A — register a box (only if not already registered)


curl -X POST https://api.trillboards.com/v1/partner/device \
  -H "Authorization: Bearer trb_partner_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"device_id":"DD-H80F55","name":"Store #123 — Front Counter","location":"123 Main St, Dallas, TX"}'

The response binds the box to a screen+venue. Send the registration device_id and use that same identifier as deviceId in Step 3. The stored binding remains external_id. (Bulk: loop your CSV of boxes.)

Done criteria (stop here when all true)