Chef Antonio

The rise of Artificial Intelligence (AI) has transformed industries, with applications ranging from security to voice assistants. Machine learning and deep learning drive these advancements, aiming to replicate human cognition and behavior. This project focuses on conversational user experience (UX) for voice-based AI products like virtual assistants, which have become adept at tasks like scheduling appointments. By leveraging collaboration between humans and machines, we aim to develop a voice agent to teach users three new cooking skills, delivering a voice-driven conversational experience tailored to targeted users.

As the use of voice-enabled devices and conversational agents like Alexa and Google Assistant continues to rise globally, the field of Human-Computer Interaction (HCI) faces the challenge of enhancing user experience with these technologies. Despite the widespread adoption of voice assistants, existing systems often fall short in providing human-like conversational interactions, instead emphasizing transactional tasks over social engagement. HCI scholars are tasked with understanding how users integrate these agents into their daily lives and addressing the limitations in their conversational capabilities. Our project seeks to tackle these challenges by incorporating psychological principles to improve the interaction between users and voice agents, particularly in the role of a coach, aiming to create a more interactive, personalized, and natural conversational experienc

The goal of this project is to design a versatile voice assistant agent tailored for cooking, specifically focusing on pasta recipes. We aim to create a highly flexible interface that caters to the diverse preferences and needs of users, distinguishing our agent as unique and exceptional among existing solutions. By prioritizing user interaction and engagement, we aspire to provide an unparalleled experience, offering authentic Italian recipes that embody the essence of culinary excellence.

In the initial phase, we established guidelines to enhance our grasp of the project and our direction. Our first objective was to clarify the user's perspective on the system and the system's view of the user. Following this, we outlined technical terms and conducted a preliminary review of the process to ensure alignment with user requirements. Subsequently, we evaluated our dialogue flow and the tools intended for user use. Throughout each step, we delineated a set of desirable skills for our agent.

In the project's next phase, we thoroughly detailed the user profile, considering their archetype, context, time limitations, and situational factors. This deep understanding enabled us to identify precise user needs, laying a strong foundation for our design process. Simultaneously, we honed in on defining our voice agent, from selecting a fitting name to crafting a concise persona, ensuring its role as a coach. We meticulously designed its behavior and personality traits to align with user expectations and promote effective interaction.

In the project's development phase, we streamlined teamwork with a collaborative model, fostering communication and synergy. We clarified our project's framework through a conceptual model, outlining key components and relationships. This guided the creation of an initial dialogue tree for user interactions. We also curated chef videos to validate recipe accuracy, enhancing the project's effectiveness and user experience.

Throughout the project's development, we engaged in iterative steps to refine our dialogue flow and user experience. Initially, we role-played our flow to simulate user interactions, taking detailed notes to identify areas for improvement. Subsequently, we enhanced the flow based on our observations, ensuring coherence and user-friendliness. We meticulously delineated the flow for each scenario and task, such as making dough, preparing pesto sauce, and compiling a shopping list. As challenges arose, such as accurately communicating measurements and conveying information reliant on visual cues, we brainstormed solutions collaboratively. By integrating descriptive language and exploring alternative communication methods, we aimed to address these challenges effectively. Through continued role-playing and refinement, we iteratively improved our dialogue flow until achieving seamless interaction and optimal user engagement.

After defining our dialogue flow, we developed a thorough evaluation framework. This included creating a table with criteria for evaluating user and agent responses. By categorizing and interpreting these responses, we aimed to assess dialogue effectiveness. This systematic approach allowed us to accurately evaluate our conversational system's performance and identify areas for improvement.

Following a thorough evaluation process, we reached the crucial stage of finalizing our dialogue flow. Through meticulous scrutiny and iterative refinement, we honed our conversational structure to its optimal form. This involved consolidating insights gathered from user testing, feedback loops, and internal evaluations. By carefully considering various factors such as user needs, system capabilities, and task requirements, we ensured that our dialogue flow was coherent, intuitive, and effectively addressed the project's objectives. This finalization marked a significant milestone in the project's development, setting the stage for further implementation and refinement efforts

Working remotely across different time zones. Approach: Implementing four options for the agent - making dough, cutting and shaping dough, boiling, and making pesto sauce. Difficulty: Required many intents and a framework to track user progress and conversation flow. Testing Challenge: If errors occurred during testing, we had to restart the entire process. Solution: Combined all dialog trees to allow users to choose options from any part of the conversation

Accent: Difficulty integrating different accents due to variations in utterances. Specific Measurement: Avoided providing specific measurements to simplify the platform; instead, the agent recommends options for users to choose based on their needs, considering factors like serving size and equipment capacity. Development: Writing intents' training phrases independently creates challenges, especially when intents share activation phrases but belong to different contexts. Updating phrases requires manual entry or downloading and updating the JSON file. Predicting all training phrases accurately is difficult, as Dialog Flow AI lacks advanced predictive capabilities.

The project aimed to create and execute a Voice Agent that would introduce and instruct users on three new cooking skills, tailored to function as either a buddy, coach, or assistant. Adhering closely to the project guidelines, we endeavored to ensure our system was user-friendly, with a particular emphasis on embodying the role of a coach. Following numerous rounds of testing and refinement, it is evident that our resources and design are effective in assisting novice users in efficiently learning pasta cooking skills.