Workshop 3: Match Maker: swipe in AR
Workshop 3: Match Maker: swipe in AR
Continue workshop 2. Add an AR icon beside the current profile on Ontdek. Tap it to open an AR page with that same profile. Scan the Match Maker card to see the person's name, photo and interests. Tap the tea cup to like or the shoe to skip. Unity saves the swipe in Firestore, Flutter shows a snackbar, and Unity displays the next profile.
Start with MatchMaker_Unity
Clone MatchMaker_Unity and open the project in Unity Hub using the Editor version recorded in ProjectSettings/ProjectVersion.txt:
git clone https://github.com/mcloots/MatchMaker_Unity.git
Let Unity restore packages and import assets, then open SampleScene. The scene and dependencies are already prepared up to the point where you add FirestoreRest.cs. The numbered workshop starts with that script.
What is already in the Unity project?
| Part | What it does | Why it is there |
|---|---|---|
ARCamera and Vuforia | Show the live camera and recognise the target. | Anchor the AR experience to a physical card. |
ImageTarget | Tracks the Match Maker card, with a width of 0.15 m. | Position the profile and choices relative to the card. |
tea and shoe | Appear as children of the recognised target. | Use them as the like/skip choices once you add the scripts. |
ProfileCanvas | A world-space UI with Background, PhotoFrame/Photo, NameLabel and InterestsLabel. | Display the profile data you will receive from Flutter. |
Photo / RawImage and AspectRatioFitter | Reserve space for a downloaded profile photo and preserve its proportions. | Show each person's public image URL without stretching it. |
NameLabel and InterestsLabel / TMP | Provide the text fields for the current profile. | Update their contents from the controller. |
DefaultObserverEventHandler | Show/hide the target's content as tracking changes. | Make the profile and models follow image recognition. |
FlutterBridge | An empty, active GameObject at the scene root. | Receive messages even when the target is out of view; attach your controller here later. |
| Newtonsoft JSON | Restored through the package references when opening the clone. | Build and read JSON messages and Firestore requests. |
| FlutterEmbed | Included project assets with SendToFlutter and the export menu. | Embed Unity in Flutter and exchange messages. |
| New Input System | Configured for the new input backend. | Detect model taps with EnhancedTouch and Editor clicks with Mouse. |
The layout and recognition already work; profile loading and swipe behaviour are what you implement in this workshop. The tea cup means like and the shoe means skip.
The hierarchy is ready for the scripts:
SampleScene
βββ Directional Light
βββ Global Volume
βββ ARCamera
βββ ImageTarget
β βββ tea
β βββ shoe
β βββ ProfileCanvas
β βββ Background
β βββ PhotoFrame
β β βββ Photo
β βββ NameLabel
β βββ InterestsLabel
βββ FlutterBridge
Use the Match Maker target JPEG (826 kB) at 15 Γ 15 cm. Recognising it already displays the models and profile layout. The scripts below will fill that layout with real profile data.
Use Flutter integration as the reference for the tested versions and Android configuration.
Keep the Match Maker app, Firebase login, repositories and four bottom-navigation items from workshop 2: Ontdek, Likes, Chats and Profiel. You add an icon to the swipe page; AR opens as a separate route with a back button.
Work in the existing Flutter project containing lib/pages/discover.dart, lib/models/profile.dart and lib/repositories/swipe_repository.dart. Use its completed workshop 2 implementation as your starting point.
Follow steps 1β7. Each complete code block identifies the file to create. The snippet in step 6 is a small addition to the existing DiscoverPage.
Keep the responsibilities simple
| Flutter | Unity |
|---|---|
| Read/filter profiles using the existing workshop 2 streams. | Receive the profile data as JSON and display it on the image target. |
| Open AR with the visible profile first, followed by the remaining unseen profiles. | Detect taps on the cup or shoe and move through that profile list. |
| Supply the signed-in user's fresh Firebase ID token when asked. | Save each swipe directly to Firestore from C#. |
| Receive a confirmed swipe event and show a snackbar. | Send confirmation only after Firestore accepts the write. |
Flutter swipe page Unity AR
AR icon β open route
current profile + remaining profiles β LoadProfiles(JSON)
scan β name, photo, interests
tap cup or shoe
β choice
fresh Firebase ID token β SaveSwipe(JSON)
C# β Firestore: save liked + timestamp
β swiped β Flutter snackbar
show next profile
Flutter already has the profiles, so pass their ID, name, photo URL and interests to Unity. Unity downloads the photo from its public URL; you do not send image bytes. In this version, Flutter reads the profiles; Unity writes the swipes.
The list is a snapshot taken when you tap the AR icon. It starts with the same profile as the regular swipe card. Return to Ontdek and reopen AR to pick up new profiles or changes. Already saved swipes are filtered out by workshop 2.
| 3D object | Meaning | Firestore field |
|---|---|---|
| Tea cup / coffee | Like | liked: true |
| Shoe | Skip | liked: false |
Files you add or change
| Project | File | Step |
|---|---|---|
| Unity | Assets/Scripts/FirestoreRest.cs | 1: write a swipe through REST. |
| Unity | Assets/Scripts/ArSwipeChoice.cs | 2.2: identify the model's choice. |
| Unity | Assets/Scripts/MatchMakerBridge.cs | 2.3: receive profiles, display them and save choices. |
| Flutter | lib/pages/ar_discover.dart | 5: open Unity, send profiles and receive events. |
| Flutter | lib/pages/discover.dart | 6: add an AR icon beside the current card. |
Firebase starting point
Use your Match Maker Firebase project and the profiles, login and swipe streams from workshop 2. Keep some unseen profiles available: reseeding does not clear swipe history, so use another app account if you have already reviewed everything.
Firestore stays fully open for this workshop:
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match /{document=**} {
allow read, write: if true;
}
}
}
The scripts save liked and createdAt under users/{uid}/swipes/{profileId}, exactly as in workshop 2. The UID organises choices per account; these open rules do not enforce ownership or expire. Flutter still supplies a user ID token for the C# request. Keep tokens out of Git and logs.
1. Add the Firestore write helper
Create Assets/Scripts/FirestoreRest.cs. This regular C# class needs no component on a GameObject. It writes to the same swipe document as workshop 2: users/{uid}/swipes/{profileId}.
using System;
using System.Collections;
using System.Text;
using Newtonsoft.Json.Linq;
using UnityEngine.Networking;
public static class FirestoreRest
{
public static IEnumerator SaveSwipe(string projectId, string uid,
string profileId, bool liked, string idToken,
Action success, Action<string> failure)
{
string root = "projects/" + projectId + "/databases/(default)/documents";
var body = new JObject {
["writes"] = new JArray {
new JObject {
["update"] = new JObject {
["name"] = root + "/users/" + uid + "/swipes/" + profileId,
["fields"] = new JObject {
["liked"] = new JObject { ["booleanValue"] = liked }
}
},
["updateMask"] = new JObject {
["fieldPaths"] = new JArray("liked")
},
["updateTransforms"] = new JArray {
new JObject {
["fieldPath"] = "createdAt",
["setToServerValue"] = "REQUEST_TIME"
}
}
}
}
};
string url = "https://firestore.googleapis.com/v1/" + root + ":commit";
using (var request = new UnityWebRequest(url, "POST"))
{
request.uploadHandler = new UploadHandlerRaw(
Encoding.UTF8.GetBytes(body.ToString()));
request.downloadHandler = new DownloadHandlerBuffer();
request.timeout = 20;
request.SetRequestHeader("Content-Type", "application/json");
request.SetRequestHeader("Authorization", "Bearer " + idToken);
yield return request.SendWebRequest();
if (request.result == UnityWebRequest.Result.Success) success();
else failure("Save failed (HTTP " + request.responseCode
+ "). Check connection, login and Firestore rules.");
}
}
}
See the JSON produced by JObject and JArray
For example, with projectId = "match-maker-demo", uid = "user-123", profileId = "demo-001" and liked = true, body.ToString() produces:
{
"writes": [
{
"update": {
"name": "projects/match-maker-demo/databases/(default)/documents/users/user-123/swipes/demo-001",
"fields": {
"liked": {
"booleanValue": true
}
}
},
"updateMask": {
"fieldPaths": ["liked"]
},
"updateTransforms": [
{
"fieldPath": "createdAt",
"setToServerValue": "REQUEST_TIME"
}
]
}
]
}
Read the C# construction and JSON side by side:
| C# | Result in JSON |
|---|---|
new JObject { ... } | An object with named properties: { ... }. |
["writes"] = ... | A property named "writes". The brackets are C# indexer syntax, not a JSON array. |
new JArray { new JObject { ... } } | An array containing an object: [{ ... }]. Here, writes contains one write operation. |
new JArray("liked") | An array containing one string: ["liked"]. |
["booleanValue"] = liked | A JSON boolean: "booleanValue": true, without quotes around true. |
body.ToString() | Converts the constructed JSON tree into the text sent in the HTTP request. |
The nested JObject values create update, fields, liked and updateMask. The JArray values create the lists writes, fieldPaths and updateTransforms. Firestore REST requires the booleanValue wrapper to identify the field's type. REQUEST_TIME asks Firestore to fill createdAt with the server timestamp; it is not a literal string stored in that timestamp field. Choosing the shoe produces the same structure with "booleanValue": false.
The Firestore REST API accepts the Firebase user ID token and checks your security rules. The commit operation writes liked and a server timestamp, matching workshop 2's FieldValue.serverTimestamp().
Checkpoint 1: save the file and check that Unity reports no compile errors. This helper runs only when the controller calls it in step 2.
2. Make the tea cup and shoe save swipes
2.1 Add a collider to tea and shoe
- Select
ImageTarget β teaand add a BoxCollider. If the object already has a suitable collider, use that one. Leave Is Trigger off. Use Edit Collider in the Inspector to fit the box around the visible cup. - Select
ImageTarget β shoeand add or fit a BoxCollider in the same way. Fit each collider to its actual model: collider dimensions use the object's local scale, so do not copy a fixed size between differently scaled models. - Keep a clear gap between the boxes. No Rigidbody is needed. Do not add a collider to the profile card: it could intercept model taps.
- Optionally add small 3D TMP labels, COFFEE Β· LIKE and SHOE Β· SKIP, under
ImageTarget, with local rotation(90,0,0).
2.2 Give each choice a meaning
Create Assets/Scripts/ArSwipeChoice.cs:
using UnityEngine;
public class ArSwipeChoice : MonoBehaviour
{
public bool liked;
}
Attach ArSwipeChoice directly to tea and shoe. In the Inspector, enable Liked on tea and leave it disabled on shoe.
Checkpoint 2.2: tea and shoe each have a BoxCollider and an ArSwipeChoice; coffee means true and shoe means false. The models do not save anything yet.
2.3 Receive the profiles and control the experience
Create Assets/Scripts/MatchMakerBridge.cs with this complete script. Attach it to the root FlutterBridge object. Assign its Inspector references in step 3 before entering Play mode.
using System;
using System.Collections;
using System.Text.RegularExpressions;
using Newtonsoft.Json.Linq;
using TMPro;
using UnityEngine;
using UnityEngine.InputSystem;
using UnityEngine.InputSystem.EnhancedTouch;
using UnityEngine.Networking;
using UnityEngine.Scripting;
using UnityEngine.UI;
using Touch = UnityEngine.InputSystem.EnhancedTouch.Touch;
[Preserve]
public class MatchMakerBridge : MonoBehaviour
{
[SerializeField] Camera arCamera;
[SerializeField] TMP_Text nameLabel;
[SerializeField] TMP_Text interestsLabel;
[SerializeField] RawImage photo;
[SerializeField] AspectRatioFitter photoAspect;
JArray profiles;
int index;
string sessionId = "", projectId = "", uid = "";
bool tracked, showing, busy, waitingForToken, pendingLiked;
float tokenDeadline;
Texture2D downloadedPhoto;
JObject Current => profiles != null && index < profiles.Count
? (JObject)profiles[index] : null;
string CurrentId => (string)Current?["id"] ?? "";
void Awake() => ClearCard();
void OnEnable() => EnhancedTouchSupport.Enable();
void OnDisable() => EnhancedTouchSupport.Disable();
void Emit(string type, string message = "", string profileId = "", bool liked = false)
{
string json = new JObject {
["type"] = type, ["sessionId"] = sessionId,
["message"] = message, ["profileId"] = profileId, ["liked"] = liked
}.ToString();
#if UNITY_EDITOR
Debug.Log(json); // Outgoing events contain no ID tokens.
#else
SendToFlutter.Send(json);
#endif
}
[Preserve]
public void RequestReady(string newSession)
{
if (sessionId != newSession)
{
ResetExperience();
sessionId = newSession;
}
Emit("ready");
}
static bool ValidSegment(string value) =>
Regex.IsMatch(value ?? "", @"^[A-Za-z0-9_-]{1,128}$");
[Preserve]
public void LoadProfiles(string json)
{
// Never log incoming JSON or credentials.
var data = JObject.Parse(json);
if ((string)data["sessionId"] != sessionId || profiles != null) return;
string project = (string)data["projectId"];
string user = (string)data["uid"];
var items = data["profiles"] as JArray;
if (!ValidSegment(project) || !ValidSegment(user) || items == null)
{
Fail("Invalid profile message.");
return;
}
foreach (JObject item in items)
{
if (!ValidSegment((string)item["id"]))
{
Fail("Invalid profile ID.");
return;
}
}
projectId = project;
uid = user;
profiles = items;
index = 0;
if (tracked) StartCoroutine(ShowProfile());
else Emit("status", "Scan the Match Maker card");
}
public void TargetFound()
{
tracked = true;
Emit("tracking", "Target found");
if (profiles != null && !showing && !busy)
StartCoroutine(ShowProfile());
}
public void TargetLost()
{
tracked = false;
Emit("tracking", "Show the Match Maker card");
}
IEnumerator ShowProfile()
{
busy = true;
showing = true;
ClearCard();
if (Current == null)
{
nameLabel.text = "All caught up!";
interestsLabel.text = "Return to Flutter to discover more profiles.";
busy = false;
Emit("empty", "No more profiles in this AR session.");
yield break;
}
nameLabel.text = (string)Current["name"] ?? "Unnamed";
var interests = Current["interests"] as JArray;
interestsLabel.text = interests == null || interests.Count == 0
? "No interests added yet" : string.Join(" Β· ", interests.ToObject<string[]>());
string url = (string)Current["imageUrl"] ?? "";
// This URL is public: never attach the Firebase token to photo requests.
if (Uri.TryCreate(url, UriKind.Absolute, out var uri)
&& uri.Scheme == Uri.UriSchemeHttps)
{
using (var request = UnityWebRequestTexture.GetTexture(url))
{
request.timeout = 15;
yield return request.SendWebRequest();
if (request.result == UnityWebRequest.Result.Success)
{
downloadedPhoto = DownloadHandlerTexture.GetContent(request);
photo.texture = downloadedPhoto;
photo.color = Color.white;
photoAspect.aspectRatio =
(float)downloadedPhoto.width / downloadedPhoto.height;
}
}
}
// Missing or failed photos keep the coloured placeholder.
busy = false;
Emit("profileLoaded", "Tap coffee to like or the shoe to skip.", CurrentId);
}
void Update()
{
if (waitingForToken && Time.realtimeSinceStartup > tokenDeadline)
Fail("Login request timed out. Tap the model again.");
if (!tracked || !showing || busy || Current == null) return;
Vector2 point = default;
bool pressed = false;
foreach (var touch in Touch.activeTouches)
{
if (touch.phase != UnityEngine.InputSystem.TouchPhase.Began) continue;
point = touch.screenPosition;
pressed = true;
break;
}
// Use a mouse click when testing in the Editor.
if (!pressed && Mouse.current != null && Mouse.current.leftButton.wasPressedThisFrame)
{
point = Mouse.current.position.ReadValue();
pressed = true;
}
if (!pressed) return;
if (!Physics.Raycast(arCamera.ScreenPointToRay(point), out RaycastHit hit)) return;
var choice = hit.collider.GetComponentInParent<ArSwipeChoice>();
if (choice == null) return;
pendingLiked = choice.liked;
busy = waitingForToken = true; // Block the second tap immediately.
tokenDeadline = Time.realtimeSinceStartup + 30;
Emit("choice", profileId: CurrentId, liked: pendingLiked);
}
[Preserve]
public void SaveSwipe(string json)
{
var data = JObject.Parse(json);
if (!waitingForToken || (string)data["sessionId"] != sessionId
|| (string)data["profileId"] != CurrentId) return;
string token = (string)data["idToken"];
if (string.IsNullOrWhiteSpace(token)) { Fail("Sign in again."); return; }
waitingForToken = false;
StartCoroutine(CommitSwipe(token));
}
IEnumerator CommitSwipe(string token)
{
string savedId = CurrentId;
bool liked = pendingLiked;
bool failed = false;
yield return FirestoreRest.SaveSwipe(projectId, uid, savedId, liked, token,
() => { },
message => { failed = true; Fail(message); });
if (failed) yield break; // Keep the same profile available for retry.
Emit("swiped", profileId: savedId, liked: liked);
index++; // Advance only after Firestore confirms success.
yield return ShowProfile();
}
[Preserve]
public void CancelSave(string json)
{
var data = JObject.Parse(json);
if (waitingForToken && (string)data["sessionId"] == sessionId
&& (string)data["profileId"] == CurrentId)
Fail("Could not get a Firebase token. Check connection and login.");
}
void Fail(string message)
{
busy = waitingForToken = false;
Emit("error", message);
}
void ClearCard()
{
photo.texture = null;
photo.color = new Color(0.90f, 0.85f, 0.87f);
if (downloadedPhoto != null) Destroy(downloadedPhoto);
downloadedPhoto = null;
nameLabel.text = "Match Maker";
interestsLabel.text = "Scan the card to start";
}
void ResetExperience()
{
StopAllCoroutines();
profiles = null;
index = 0;
showing = busy = waitingForToken = false;
projectId = uid = "";
ClearCard();
// Keep tracked: the card may still be visible when AR reopens.
}
[Preserve]
public void ClearSession(string expectedSession)
{
if (expectedSession != sessionId) return;
ResetExperience();
sessionId = "";
}
void OnDestroy()
{
if (downloadedPhoto != null) Destroy(downloadedPhoto);
}
}
Unity receives ordinary JSON objects such as {"id":"demo-001","name":"Lina","imageUrl":"https://β¦","interests":["Coffee","Music"]}. There is no Firestore profile query or typed stringValue conversion in this controller. It uses the data Flutter sends.
The session ID separates visits to the AR page. The current profile ID identifies the pending choice. Together they stop a delayed token response from saving the wrong profile after leaving and reopening AR.
The controller uses the new Input System's EnhancedTouch API for Android touches and Mouse.current for Editor clicks. OnEnable and OnDisable balance enhanced-touch recording. No Input Actions asset is needed for these simple taps. See Unity's touch input guide.
3. Wire the Inspector and tracking events
Assign the five Inspector references on FlutterBridge:
| Field | Scene object/component |
|---|---|
| AR Camera | Camera component on ARCamera |
| Name Label | TMP component on NameLabel |
| Interests Label | TMP component on InterestsLabel |
| Photo | RawImage on Photo |
| Photo Aspect | AspectRatioFitter on Photo |
On ImageTarget, connect On Target Found and On Target Lost from DefaultObserverEventHandler to FlutterBridge β TargetFound() and TargetLost(). Vuforia controls the visibility of child renderers, canvases and colliders; the script also checks tracked before accepting a tap. See Vuforia's event-handler documentation.
Taps use a ray from the AR camera through the screen point and Physics.Raycast. The hit object's ArSwipeChoice determines the boolean. Profile photos use UnityWebRequestTexture.GetTexture and are destroyed when the next profile replaces them.
Why do we use [Preserve]?
[Preserve] is a C# attribute from UnityEngine.Scripting. It tells Unity's linker to keep the annotated code in the built app. During a build, managed code stripping can remove code that appears unused to reduce the app size.
Flutter calls Unity methods by their names as strings, rather than through a normal C# method call:
sendToUnity('FlutterBridge', 'LoadProfiles', json);
Unity's build analysis cannot always detect these calls. Mark the receiving method so it remains available in the Android build:
[Preserve]
public void LoadProfiles(string json)
{
// The existing method body receives the profiles from Flutter.
}
This is why the bridge's callable methods, such as RequestReady, LoadProfiles and SaveSwipe, have [Preserve]. Without that protection, a method can work in the Editor but be removed from a build, depending on the stripping settings.
The attribute on the class preserves the type and its default constructor; it does not automatically preserve every method. Keep the method attributes too. See Unity's code preservation documentation.
The SaveSwipe message contains a token; never log it. Check the session and profile IDs before applying a token response to the current AR page.
Follow the simple round trip
LoadProfilesreceives the list from Flutter;ShowProfileshows its first item.Updatefinds the tapped model through a raycast and sendschoiceto Flutter.- Flutter obtains a fresh token and calls Unity's
SaveSwipe. CommitSwipecalls the C# Firestore helper.- After success, Unity emits
swiped, incrementsindexand shows the next item. - Flutter receives
swipedand displays a snackbar.
The extra token exchange keeps login in Flutter; you do not implement a second login in Unity or pass a service-account key.
Checkpoint 3: the scene compiles, all five Inspector fields are assigned and the models/card follow the marker. You will pass real profile data after adding the Flutter route in step 5.
Messages in both directions
| Direction | Message | Contents |
|---|---|---|
| Flutter β Unity | RequestReady | Session ID; check that the scene receiver exists. |
| Unity β Flutter | ready | Send the profile list now. |
| Flutter β Unity | LoadProfiles | Session ID, project ID, UID and profile objects. |
| Unity β Flutter | choice | Current profile ID and liked boolean; ask Flutter for a token. |
| Flutter β Unity | SaveSwipe | Session ID, profile ID and fresh ID token. |
| Unity β Flutter | swiped | Confirmed profile ID and liked boolean; show the snackbar. |
| Unity β Flutter | profileLoaded / tracking / empty / error | Update status text. |
| Flutter β Unity | CancelSave / ClearSession | Release a failed token request or close the session. |
Flutter uses sendToUnity("FlutterBridge", method, json). Unity replies through SendToFlutter.Send(json), received by onMessageFromUnity. See the embedding plugin's message APIs.
4. Apply the existing Android integration instructions
Follow Flutter integration for the Android settings in both Unity and Flutter, plugin setup, Unity export and Android files. Use the tested versions listed on that page.
Apply that guide to the projects you already have:
Open your Unity project and add the existing
SampleSceneto the enabled build scenes. Export this scene with your new components and profile UI.Keep the existing Match Maker app from workshop 2. Do not create a new Flutter app or replace its Firebase
main.dart, theme, application ID, Google Services configuration orAuthGate.Export the completed Unity scene into that app's
android/unityLibrary.Apply the guide's Unity Player Settings, export checker change, Android SDK/NDK/Gradle settings, MainActivity and other Android integration settings.
Make sure the final Android manifest includes internet and camera permissions. The REST helper and profile photos need internet access; Vuforia needs the camera.
Add
flutter_embed_unityto the existingpubspec.yamland runflutter pub get:dependencies: flutter: sdk: flutter cupertino_icons: ^1.0.8 firebase_core: ^5.1.0 firebase_auth: ^7.0.1 cloud_firestore: ^7.1.0 # Add this dependency: flutter_embed_unity: 2.0.0The Firebase dependencies above are already present in the Match Maker project. Add only the Unity embedding line; keep your existing dependency versions, SDK constraints, dev dependencies and assets. This snippet shows the dependencies section, not a replacement for the entire pubspec.
Open
android/app/src/main/AndroidManifest.xmlin the Flutter project. Add this directly inside<application>, alongside<activity>:<meta-data android:name="io.flutter.embedding.android.EnableImpeller" android:value="false" />
Use the FlutterEmbed files already included in the Unity repository when following the integration guide. Add only flutter_embed_unity to the Flutter app's pubspec.yaml; its Android implementation is resolved automatically.
What changes when you disable Impeller?
Impeller is Flutter's rendering engine: it draws the Flutter interface. On supported Android devices, a regular build uses it by default. Setting EnableImpeller to false selects Flutter's older rendering backend. Unity continues to render its own scene with its configured graphics API.
In our tested setup, the standalone Unity app worked, but the embedded camera view stayed black with Impeller. Disabling it made the embedded AR view work. This is a compatibility setting for this integration.
| How you run/build | Rendering behaviour |
|---|---|
| Regular build without this manifest setting | Uses the default Flutter renderer, normally Impeller on supported Android devices. |
flutter run --no-enable-impeller | Disables Impeller for that run. |
| Build with the manifest setting above | Stores the preference in the Android app, including debug and release builds, so a special run command is unnecessary. |
This setting changes the renderer, not the build mode: debug is still debug and release is still release. Stop the app and rebuild with flutter run after editing the manifest; hot reload does not apply this change. See the official Flutter explanation and the integration guide.
Checkpoint 4: Android configuration is complete and unityLibrary exists in your Match Maker app. The AR icon is added in step 6, so the existing app still opens its regular screens at this point.
After changing a Unity scene or script, export again, stop the running Flutter app and fully rebuild. Hot reload does not update the exported Unity library. Use a physical Android device for this workshop.
5. Create the Flutter AR page
Create lib/pages/ar_discover.dart. Give the page a list of the already filtered Profile objects. Keep their order: the current swipe card must be first.
Reuse lib/models/profile.dart as it is. Its id, name, imageUrl and interests fields match the JSON sent by _sendProfiles below. The model's toMap() omits the document ID, so include profile.id explicitly in the message. You do not need another profile model or changes to the repositories.
import 'dart:async';
import 'dart:convert';
import 'package:firebase_auth/firebase_auth.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:flutter/material.dart';
import 'package:flutter_embed_unity/flutter_embed_unity.dart';
import '../models/profile.dart';
class ArDiscoverPage extends StatefulWidget {
const ArDiscoverPage({super.key, required this.profiles});
final List<Profile> profiles;
State<ArDiscoverPage> createState() => _ArDiscoverPageState();
}
class _ArDiscoverPageState extends State<ArDiscoverPage> {
final _sessionId = DateTime.now().microsecondsSinceEpoch.toString();
Timer? _readyTimer;
StreamSubscription<User?>? _authSubscription;
String? _uid;
bool _ready = false;
bool _ended = false;
String _status = 'Starting Unity...';
String _tracking = 'Show the Match Maker card';
void initState() {
super.initState();
_uid = FirebaseAuth.instance.currentUser?.uid;
_authSubscription = FirebaseAuth.instance.authStateChanges().listen((user) {
if (mounted && !_ended && user?.uid != _uid) {
_ended = true;
_readyTimer?.cancel();
sendToUnity('FlutterBridge', 'ClearSession', _sessionId);
Navigator.of(context).pop();
}
});
_readyTimer = Timer.periodic(const Duration(seconds: 1), (_) {
if (!_ready) sendToUnity('FlutterBridge', 'RequestReady', _sessionId);
});
}
void _sendProfiles() {
sendToUnity('FlutterBridge', 'LoadProfiles', jsonEncode({
'sessionId': _sessionId,
'projectId': Firebase.app().options.projectId,
'uid': _uid,
'profiles': widget.profiles.map((profile) => {
'id': profile.id,
'name': profile.name,
'imageUrl': profile.imageUrl,
'interests': profile.interests,
}).toList(),
}));
}
Future<void> _authoriseSwipe(String profileId) async {
try {
final user = FirebaseAuth.instance.currentUser;
if (user == null || user.uid != _uid) throw StateError('Sign in again.');
final token = await user.getIdToken(true);
if (!mounted || _ended) return;
if (token == null || FirebaseAuth.instance.currentUser?.uid != _uid) {
throw StateError('Sign in again.');
}
// Unity performs the write; Flutter supplies only the user credential.
sendToUnity('FlutterBridge', 'SaveSwipe', jsonEncode({
'sessionId': _sessionId,
'profileId': profileId,
'idToken': token,
}));
} catch (_) {
if (!mounted || _ended) return;
sendToUnity('FlutterBridge', 'CancelSave', jsonEncode({
'sessionId': _sessionId,
'profileId': profileId,
}));
}
}
void _onUnityMessage(String raw) {
if (!mounted || _ended) return;
Map<String, dynamic> event;
try {
event = jsonDecode(raw) as Map<String, dynamic>;
} catch (_) {
return;
}
if (event['sessionId'] != _sessionId) return;
switch (event['type']) {
case 'ready':
if (_ready) return;
_readyTimer?.cancel();
setState(() {
_ready = true;
_status = 'Scan the card to show the profile.';
});
_sendProfiles();
break;
case 'choice':
final profileId = event['profileId'];
if (profileId is! String) return;
setState(() => _status = 'Saving swipe...');
_authoriseSwipe(profileId);
break;
case 'swiped':
final message = event['liked'] == true ? 'Profiel geliket!' : 'Profiel overgeslagen.';
ScaffoldMessenger.of(context)
..hideCurrentSnackBar()
..showSnackBar(SnackBar(
content: Text(message),
duration: const Duration(seconds: 2),
));
setState(() => _status = 'Loading next profile...');
break;
case 'tracking':
setState(() => _tracking = event['message'] as String? ?? '');
break;
case 'profileLoaded':
case 'status':
case 'empty':
case 'error':
setState(() => _status = event['message'] as String? ?? '');
break;
}
}
void dispose() {
_ended = true;
_readyTimer?.cancel();
_authSubscription?.cancel();
sendToUnity('FlutterBridge', 'ClearSession', _sessionId);
super.dispose();
}
Widget build(BuildContext context) => Scaffold(
appBar: AppBar(title: const Text('Swipe in AR')),
body: SafeArea(
child: Column(
children: [
Expanded(child: EmbedUnity(onMessageFromUnity: _onUnityMessage)),
Padding(
padding: const EdgeInsets.all(12),
child: Column(
mainAxisSize: MainAxisSize.min,
children: [Text(_tracking), Text(_status)],
),
),
],
),
),
);
}
Point out the three connections:
_sendProfilessends the existing Flutter data to Unity._authoriseSwipesupplies a fresh token. The Firestore write stays in Unity C#.- The
swipedcase receives a confirmed save and shows the snackbar.
The readiness request repeats until Unity responds. This also works when you go back and reopen AR: the Unity runtime may already be loaded. The route owns its Scaffold, app bar and back button. Leaving it clears the scene and disposes EmbedUnity.
Checkpoint 5: the new file has no import or syntax errors. Connect it to the current card in the next step.
6. Add an AR icon beside the current swipe card
Open lib/pages/discover.dart and make these three additions.
6.1 Add two imports at the top of the file
Add these lines to the imports:
import 'package:flutter/foundation.dart';
import 'ar_discover.dart';
6.2 Add _openAr inside _DiscoverPageState
Find class _DiscoverPageState extends State<DiscoverPage>. Add the following method inside that class, directly before _swipe(Profile profile, bool liked). Place it outside the build method:
void _openAr(List<Profile> profiles) {
if (isSaving || profiles.isEmpty) return;
if (kIsWeb || defaultTargetPlatform != TargetPlatform.android) {
ScaffoldMessenger.of(context).showSnackBar(
const SnackBar(content: Text('Open AR op een Android-toestel.')),
);
return;
}
Navigator.of(context).push<void>(
MaterialPageRoute<void>(
builder: (_) => ArDiscoverPage(
profiles: List<Profile>.unmodifiable(profiles),
),
),
);
}
6.3 Insert the AR icon before the swipe card
- In
build, search forfinal profile = profiles.first;inside the nested StreamBuilders. - Immediately below it, find
return SingleChildScrollView(...)and itschild: Column(...). - In that Column's
children: [...]list, insert the following as the first widget, directly beforeDismissible(...):
Align(
alignment: Alignment.centerRight,
child: IconButton(
tooltip: 'Bekijk dit profiel in AR',
icon: const Icon(Icons.view_in_ar),
onPressed: isSaving ? null : () => _openAr(profiles),
),
),
This places the icon above the card on the right. _openAr(profiles) passes the current profile first, followed by the remaining unseen profiles.
Checkpoint 6: tap the AR icon. A separate page opens. Scan the card and check that its first name, photo and interests match the profile you just saw in Ontdek. Tap coffee: a snackbar appears after saving and the next profile appears in AR. Use the back arrow to return to the ordinary swipe page.
6.4 Follow the communication in order
Follow one complete visit to AR. These are excerpts from the code above to explain the sequence; you do not add these snippets again.
1. Firestore β Flutter: load profiles and previous swipes
The workshop 2 repositories listen to Firestore. DiscoverPage filters out reviewed profiles before you open AR:
final profilesStream = ProfileRepository().watchProfiles();
final swipesStream = repository.watchSwipes();
The filtered profiles list has the current swipe card first. Unity has not received anything yet.
2. Flutter β the AR route: pass the current profile and the remaining list
Tap the AR icon. _openAr opens the page with a stable copy of that list:
builder: (_) => ArDiscoverPage(
profiles: List<Profile>.unmodifiable(profiles),
),
This is ordinary Flutter navigation, not a Unity message. The next steps transfer the data into the embedded Unity scene.
3. Flutter β Unity: ask whether the scene is ready
The timer calls the public method RequestReady on the GameObject named FlutterBridge. It repeats until Unity answers:
sendToUnity('FlutterBridge', 'RequestReady', _sessionId);
4. Unity β Flutter: answer ready
RequestReady records the session and sends an event back:
Emit("ready");
Emit builds the JSON event. On the Android device it sends that string through the plugin:
SendToFlutter.Send(json);
Flutter receives it through EmbedUnity(onMessageFromUnity: _onUnityMessage). The ready case stops the timer and calls _sendProfiles().
5. Flutter β Unity: send the profile data
_sendProfiles serializes the list and calls LoadProfiles in Unity:
sendToUnity('FlutterBridge', 'LoadProfiles', jsonEncode({
'sessionId': _sessionId,
'projectId': Firebase.app().options.projectId,
'uid': _uid,
'profiles': widget.profiles.map((profile) => {
'id': profile.id,
'name': profile.name,
'imageUrl': profile.imageUrl,
'interests': profile.interests,
}).toList(),
}));
Unity now has the same profile data as Flutter. There is no additional Firestore profile read from Unity. The photo URL points to the public image host.
6. Unity β Flutter: report tracking and the displayed profile
Scan the image target. Unity shows the first profile and sends status events:
Emit("tracking", "Target found");
// After the profile and its photo attempt are ready:
Emit("profileLoaded", "Tap coffee to like or the shoe to skip.", CurrentId);
Flutter displays these messages in its tracking/status text. Unity downloads the photo directly from imageUrl; that download is separate from Firestore and contains no Firebase token.
7. Unity β Flutter: report the tapped choice
Tap tea or shoe. Unity remembers the boolean, blocks another tap and asks Flutter to authorise the save:
pendingLiked = choice.liked;
busy = waitingForToken = true;
Emit("choice", profileId: CurrentId, liked: pendingLiked);
choice means a choice was made, not that it was saved. Flutter does not show the success snackbar yet.
8. Flutter β Unity: supply a fresh user token
The choice case calls _authoriseSwipe(profileId). Flutter obtains the token from Firebase Authentication, then returns it to Unity:
final token = await user.getIdToken(true);
sendToUnity('FlutterBridge', 'SaveSwipe', jsonEncode({
'sessionId': _sessionId,
'profileId': profileId,
'idToken': token,
}));
Getting this token is not a Firestore write. Unity checks the session/profile IDs and calls CommitSwipe using the choice it remembered in step 7.
9. Unity β Firestore: save the swipe from C#
FirestoreRest.SaveSwipe sends an authenticated REST commit:
using (var request = new UnityWebRequest(url, "POST"))
{
request.SetRequestHeader("Authorization", "Bearer " + idToken);
// The full helper also attaches the JSON write body and handlers.
yield return request.SendWebRequest();
}
The write targets users/{uid}/swipes/{profileId}. The JSON body sets liked to the selected boolean and createdAt to a server timestamp. Flutter does not call repository.saveSwipe for this AR choice.
10. Firestore β Unity: confirm the write
The REST helper checks the HTTP result before reporting success:
if (request.result == UnityWebRequest.Result.Success) success();
else failure("Save failed (HTTP " + request.responseCode + ").");
On failure, Unity keeps the current profile available and sends an error event. On success, the controller continues with the next step.
11. Unity β Flutter: confirm the saved swipe and advance
Unity sends swiped, then advances its own profile index:
Emit("swiped", profileId: savedId, liked: liked);
index++;
yield return ShowProfile();
Flutter handles swiped by showing the snackbar:
final message = event['liked'] == true ? 'Profiel geliket!' : 'Profiel overgeslagen.';
ScaffoldMessenger.of(context).showSnackBar(
SnackBar(content: Text(message)),
);
Unity shows the next item from the list it already received, then sends another profileLoaded event. The snackbar and the next profile can appear almost simultaneously; Unity does not wait for the snackbar to disappear. Repeat steps 7β11 for each profile.
12. Firestore β Flutter: update the ordinary swipe page
The existing swipe listener also receives the saved document:
return swipes.snapshots().map((snapshot) => {
for (final doc in snapshot.docs) doc.id: doc.data()['liked'] == true,
});
Ontdek filters out that profile and Likes includes it when liked is true. This listener runs independently of the Unity bridge: its update can arrive before or after swiped. You do not manually save the same choice again in Flutter.
13. Flutter β Unity: clean up when returning
Press Back. The AR page's dispose clears the session:
sendToUnity('FlutterBridge', 'ClearSession', _sessionId);
Unity cancels local work and clears the profile/photo. Flutter returns to the swipe page, whose existing streams reflect the saved choices.
Other messages during the flow
| When | Direction and small code excerpt | Result |
|---|---|---|
| The marker disappears | Unity β Flutter: Emit("tracking", "Show the Match Maker card"); | Update the guidance; taps are blocked until tracking returns. |
| Token retrieval fails | Flutter β Unity: sendToUnity('FlutterBridge', 'CancelSave', jsonEncode({'sessionId': _sessionId, 'profileId': profileId})); | Release the pending choice so the same profile can be tried again. |
| The Firestore write fails | Unity β Flutter: Emit("error", message); | Show the error in the status text; no success snackbar and no advance. |
| The list is exhausted | Unity β Flutter: Emit("empty", "No more profiles in this AR session."); | Show the empty state; return to Flutter to open a new list. |
7. Test the complete flow
Run flutter analyze and flutter run on a physical Android device.
| Test | Expected result |
|---|---|
| Inspect Match Maker navigation | Four original bottom-nav items; an AR icon beside the current card on Ontdek. |
| Tap AR while profile A is visible | A new route opens; scanning shows profile A with its name, photo and interests. |
| Inspect the next card after choosing | Profile B follows A in the order sent by Flutter; rescanning is unnecessary. |
| Tap coffee | C# saves liked: true plus a server timestamp; Flutter shows Profiel geliket!; Unity advances. |
| Tap the shoe | C# saves liked: false; Flutter shows Profiel overgeslagen.; Unity advances. |
| Tap rapidly while saving | Only one choice is accepted for the current profile. |
| Disconnect the network before choosing | An error appears; the same profile stays available and no success snackbar appears. Reconnect and retry. |
| Hide the marker | Content hides and taps cannot choose either model. Show it again to continue with the current profile. |
| Use a missing/broken photo URL | Name/interests still appear with a coloured photo placeholder. |
| Review the last profile | AR shows All caught up! and cannot save another swipe. |
| Press Back | Return to Ontdek. Swiped profiles are gone and coffee choices appear in Likes through the existing streams. |
| Open AR again | It starts with the newly visible card; already saved choices are filtered out by Flutter. |
| Open from Chrome | A short Android reminder appears; Unity is not opened. |
Inspect Firestore: each choice is stored under users/{signed-in-uid}/swipes/{profileId}, exactly as in workshop 2. There is no separate AR collection. With the open workshop rules, the UID in the document path organises the data; it does not restrict who can access it.
Understand this short-session example
The profile list is copied once, when opening AR. Unity advances its local index only after a confirmed write. Returning to Flutter uses the existing realtime streams to reconcile the regular swipe page.
A timed-out request may already have been accepted by Firestore. Retrying the same profile writes the same document ID; it does not create a duplicate. Leaving the route cancels local work and clears the scene, but cannot undo an accepted write. Keep this workshop session in the foreground while saving.
Troubleshooting
| Symptom | Check |
|---|---|
| Wrong first profile | Pass the filtered profiles list from beside final profile = profiles.first; preserve its order. |
| No profiles reach Unity | Exact FlutterBridge name, ready handshake and LoadProfiles payload. Export again after C# changes. |
| Camera works in standalone Unity but is black in Flutter | Disable Impeller in the Flutter Android manifest as shown in Flutter integration, then fully stop and rebuild the app. Test with flutter run --no-enable-impeller to compare. |
| Camera also fails in standalone Unity | Check the physical device, Vuforia license, camera permissions and Unity settings from Flutter integration. |
| Photo is blank | Public HTTPS URL, network access and RawImage/AspectRatioFitter references. |
| Tapping does nothing | Input System installed, Active Input Handling = Input System Package (New), EnhancedTouch enabled by the controller, ARCamera reference and colliders on tea/shoe. |
| Cup skips or shoe likes | Correct the liked checkbox on ArSwipeChoice. |
| HTTP 401/403 | Fresh user ID token, matching project/UID and published rules. If your project enforces App Check, an App Check implementation is also needed. |
| Snackbar appears before saving | Show it for swiped, not choice. |
| Changes missing on the device | Re-export Unity, stop the app and rebuild Flutter. |
Explain the round trip: where is the profile list created, how is it sent to Unity, which script writes Firestore, and which message makes Flutter show the snackbar?