We propose a car experience where users can personalize rides by choosing personas with AI for a unique and immersive Mercedes-Benz experience. Personas accompany users on their journey, like a friend who knows exactly what you’re looking for in an immersive experience.

Imagine choosing the F1 Persona: race car sound effects would amplify your ride, famous drivers could be the voice that navigates your directions and shares F1-specific news highlights, and your in-car display could present a speedometer that’s visually similar to that of a race car.

Like any friend, our system has personality. And so will your car.

Here’s a walkthrough of the data flow. To use the service, users will create a profile online through our app. Then, we’ll finetune a LLM based on this profile. Users can also opt to link their other apps (e.g. calendar, Spotify, etc) to provide more information. This personalized LLM will be connected to Mercedes’ built-in MBUX system (supports voice control) to interact with the users. To simulate “growth,” we’ll periodically update the LLMs with the requests users made, sensor data, or web data as the user spends more time in the vehicle.

To output responses in the style of the chosen persona, requests from the user will be modified in the Control Node into a prompt that can be fed into the LLM. One example of this might be “respond to the following as a F1 race engineer + user’s request”, where the persona and modifying personality attributes can be changed to match the set persona. In the case that the embedded LLM is unable to properly modify its responses to a satisfactory degree, we propose the following technical architecture:

In this version, instead of modifying the prompt to the LLM to respond as the chosen persona, we will use a second LLM specifically trained to handle translating input text into the style of the chosen persona and personality traits. As this version is more expensive from a storage and computation standpoint, we will stick to the version 3a architecture as long as the results meet a certain standard of quality.

Objective Function

• Testing fine-tuned GPT: Minimize Latency
• Response time from audio prompt to response (through model)
• Correctness of inference of the model (i.e does what it’s instructed)
• Minimal use of online queries to ChatGPT (ideally all requests are handled by the embedded LLM)
• Testing profile creation and updates:
• Number of corrections to the system (i.e negative feedback)
• User adoption:
• How many users enable the AI assistant? What percentage of the time?
• When do users utilize F1 mode?

We will continue working on this as a team until June 2024.

Team Composition:

Yannie Tan - User Experience, Front-end

Esteban Wu - Data, Front-end

Katherine Chen - User Experience, Front-end

Ryan Kang - AI, Back-end

William Fang - AI, Back-end