Abstract
<p><span style="font-weight: 400;">Soft skills are increasingly vital in AI-driven workplaces, yet their effective measurement remains challenging. This study examines WiseWorld, an AI-powered social simulation platform that assesses 44 soft skills using dynamic, story-based interactions across six categories. Data were collected from 66 participants, focusing on 52 active users engaged in at least two episodes. The reliability of WiseWorld's assessments was evaluated using the Coefficient of Variation (CV). The analysis revealed moderate variability across different episode scenarios, with a CV range of 70 to 100, indicating consistent performance. Engagement metrics—averaging 19 minutes of daily playtime with day-1 and day-3 retention rates of 63% and 41%, respectively—demonstrate the platform’s strong capacity to maintain user interest and yield meaningful skill evaluations. These findings position WiseWorld as a scalable and innovative tool that redefines how organizations measure and develop essential soft skills. It has broad implications for workforce development, offering data-driven insights to address skill gaps and enhance employee readiness in an AI-driven economy.</span></p>Introduction
<p><span style="font-weight: 400;">As workplaces increasingly rely on AI, evaluating soft skills—critical competencies like adaptability and communication—has become a significant challenge. The demand for human-centric skills has surged in industries where AI automates technical tasks. Studies indicate that organizations with high soft skills proficiency see improved team performance and innovation rates. Traditional methods, such as structured interviews or multiple-choice questionnaires, are often limited in providing genuine insights. </span><strong>Research indicates that gamification can significantly sustain user interest and productivity, making it a promising approach for soft skills assessment</strong><a href="https://games.jmir.org/2023" class="no-underline text-accent"><strong> (Altomari et al., 2023</strong></a><strong>).</strong><span style="font-weight: 400;"> </span></p>
<p><span style="font-weight: 400;">WiseWorld stands out by harnessing AI to replicate real-world interactions and evaluate user decisions across 44 soft skills. The platform's distinct features include:</span></p>
<ul>
<li style="font-weight: 400;" aria-level="1"><strong>AI Narrator: </strong><span style="font-weight: 400;">The platform features one AI storyteller—akin to a Dungeons & Dragons–style “Dungeon Master”—who adapts each scenario in real time to reflect user decisions. This personalized approach ensures that every participant encounters a unique, customized journey based on their choices and behaviors.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Unbiased, Context-Aware Assessments:</strong><span style="font-weight: 400;"> AI objectively analyzes user decisions, aligning them with situational appropriateness and skill relevance. The WiseWorld method can eliminate interviewer bias and minimize test preparation effects, providing a fair and accurate evaluation of soft skills.</span></li>
</ul>
<p><span style="font-weight: 400;">WiseWorld engages users with an open-world, story-based framework that leverages gamification principles. The map-based interface guides users through daily narrative touchpoints, offering three diverse daily challenges. These life-like episode scenarios evolve based on user actions, ensuring authentic engagement and accurate evaluation of soft skills, sparking excitement about the platform's potential.</span></p>Sample Data
<p><span style="font-weight: 400;">The study recruited 66 participants from three distinct sectors:</span></p>
<ul>
<li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">A coach training institute in the education domain,</span></li>
<li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">A fintech company, and</span></li>
<li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">An online healthcare platform that connects users to doctors via a mobile application.</span></li>
</ul>
<p><span style="font-weight: 400;">This diverse group provided a wide range of professional backgrounds and skill levels, helping to capture robust interactions with AI-driven and gamified environments. Of the 66 individuals who initially signed up,</span><strong> 52 were considered</strong><span style="font-weight: 400;"> “</span><strong>active</strong><span style="font-weight: 400;">” users—defined here as those</span><strong> who completed at least three episodes within a month</strong><span style="font-weight: 400;">. Focusing on these 52 active users ensured sufficient interaction data for more reliable analysis, striking an optimal balance between sample size and data integrity.</span></p>User Journey in WiseWorld
<p><span style="font-weight: 400;">User Journey in WiseWorld: Step-by-Step</span></p>
<ol>
<li style="font-weight: 400;" aria-level="1"><strong>Personalize Goals and Avatar (First Sign-In): </strong><span style="font-weight: 400;">The user or a manager sets the primary objectives for skill development and then creates a customized avatar (realistic or fictional) to reflect those goals.</span><img src="https://wiseworld.blob.core.windows.net/blog-section/wiseworld_old_product_character_creation.webp" class="rounded" alt="wiseworld_old_product_character_creation" width="auto" height="300" loading="lazy"></li>
<li style="font-weight: 400;" aria-level="1"><strong>Choose a Location: </strong><span style="font-weight: 400;">On the interactive map, select a spot where the challenge aligns with user-stated goals. Ensure that each scenario directly supports user skill-development objectives.</span><img src="https://wiseworld.blob.core.windows.net/blog-section/wiseworld_old_product_map.webp" class="rounded" alt="wiseworld_old_product_map" width="auto" height="300" loading="lazy"></li>
<li style="font-weight: 400;" aria-level="1"><strong>Immerse the user in the Story: </strong><span style="font-weight: 400;">View three to five narrative slides that set the scene and describe the main challenge.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Make a Decision:</strong><span style="font-weight: 400;"> Respond to prompts like "What will you do?" and choose your actions.</span><img src="https://wiseworld.blob.core.windows.net/blog-section/wiseworld_old_product_episode.webp" class="rounded" alt="wiseworld_old_product_episode" width="auto" height="300" loading="lazy"></li>
<li style="font-weight: 400;" aria-level="1"><strong>Chat with the AI character:</strong><span style="font-weight: 400;"> Users engage in a dynamic conversation with the AI storyteller to share user decisions or ask questions.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>The consequence of the User’s choice:</strong><span style="font-weight: 400;"> The AI narrator updates the story based on the user’s decisions, revealing immediate consequences and guiding the user to the next narrative stage.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Receive Skill Scores:</strong><span style="font-weight: 400;"> From a pool of 44 possible soft skills, users receive scores (ranging from –5 to +5) on the 3–7 skills most relevant to their decisions. Each score is explained by how the user's choices demonstrated—or lacked—those targeted skills.</span><img src="https://wiseworld.blob.core.windows.net/blog-section/wiseworld_old_product_result.webp" class="rounded" alt="wiseworld_old_product_result" width="auto" height="300" loading="lazy"></li>
<li style="font-weight: 400;" aria-level="1"><strong>Review Feedback: </strong><span style="font-weight: 400;">Receive concise, AI-generated insights that highlight user strengths, pinpoint areas for improvement, and offer micro-learning strategies for similar future situations.</span><img src="https://wiseworld.blob.core.windows.net/blog-section/wiseworld_old_product_power_wheel.webp" class="rounded" alt="wiseworld_old_product_power_wheel" width="auto" height="300" loading="lazy"></li>
</ol>
<p><span style="font-weight: 400;">Upon entering WiseWorld, users are invited to create a character that can mirror their real-life persona or an entirely fictional identity. This character-creation process is designed to enhance the gamification aspect of WiseWorld. However, from a soft-skills perspective, the specific identity users adopt—realistic or imaginary—does not affect their proficiency. Whether they play as themselves or as fictional characters, the AI-driven assessment focuses solely on behaviors and decisions made in the simulation. In simpler terms, it does not matter whether users role-play as themselves or a fictional character—soft skills are measured the same way regardless of the persona they choose. Drawing inspiration from role-playing games like Dungeons & Dragons (D&D), WiseWorld allows users to navigate interactive episodes categorized into three domains: work, life, and hobbies based on user goals.</span></p>
<p><span style="font-weight: 400;">Each interactive episode begins with users selecting one of up to three locations on a map, each presenting a unique challenge. Upon selecting a location, the narration unfolds through 2-3 slides, detailing the situation, the involved parties, and the main challenge. After the narration, users are prompted with questions such as, "What would you do now?" Their responses initiate AI-driven interactions, continuing for at least five rounds. The AI evaluates these responses based solely on the quality of the user’s decision, determined by how well it aligns with the target soft skill. Post-decision analysis categorizes user performance across 44 soft skills, displayed on a PowerWheel—a radar chart highlighting strengths and improvement areas. By visualizing scores intuitively, the PowerWheel enables users to quickly identify their best-performing soft skills and areas requiring development, offering a clear roadmap for personal growth.</span></p>
<p><span style="font-weight: 400;">Scores range from -5 to +5, reflecting user interactions and decisions. Positive scores indicate alignment with the target soft skill—for instance, a well-crafted response showcasing deductive reasoning would result in a higher score. Negative scores (excluding 0) signify a lack of use and actions contrary to the skill—for example, denying another's feelings instead of showing empathy. Frequent and high-quality interactions improve the precision of evaluations, while limited interactions challenge the tool’s ability to provide comprehensive insights.</span></p>Feedback Mechanism in WiseWorld
<p><span style="font-weight: 400;">After each episode, after the user engages in dialogue with the AI acting as a </span><strong>Dungeons & Dragons–style game master</strong><span style="font-weight: 400;">, the system transitions from interactive storytelling to reflective feedback. Once the user’s decisions have shaped the narrative, the AI narrates the story’s outcome based on those decisions. Following this narration, the AI provides </span><strong>personalized feedback</strong><span style="font-weight: 400;">, highlighting the strengths and areas for improvement in the user’s soft skills, as demonstrated during the episode.</span></p>
<p><span style="font-weight: 400;">This immediate, individualized feedback serves multiple purposes:</span></p>
<ul>
<li style="font-weight: 400;" aria-level="1"><strong>Reinforcement and Reflection:</strong><span style="font-weight: 400;"> It reinforces positive behaviors and provides constructive criticism, helping users understand how their choices align with adequate soft skills.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Guidance for Future Actions:</strong><span style="font-weight: 400;"> Users can apply this feedback in subsequent episodes to improve decision-making and soft skill application.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Enhanced Learning Experience:</strong><span style="font-weight: 400;"> By offering tailored insights right after each interaction, the platform supports continuous learning and self-awareness, making the assessment more impactful and actionable.</span></li>
</ul>Methodology
<h4><strong>Research Design</strong></h4>
<h4><span style="font-weight: 400;">This study employed a quantitative framework to evaluate the reliability of WiseWorld’s soft skills assessments. The primary metric chosen was the Coefficient of Variation (CV), valued for its ability to standardize score variability and facilitate comparisons across diverse user interactions. </span><strong>CV is particularly suitable for applications where score variability scales proportionally to the mean, ensuring reliability across diverse datasets (Shechtman, 2013).</strong><span style="font-weight: 400;"> This adaptability makes CV an effective measure for evaluating consistency in dynamic, user-driven environments like WiseWorld. (</span><a href="https://link.springer.com/chapter/10.1007/978-3-642-37131-8_4" class="no-underline text-accent"><span style="font-weight: 400;">Springer</span></a><span style="font-weight: 400;">)</span></h4>
<h5><strong>Why CV?</strong></h5>
<h4><span style="font-weight: 400;">The </span><strong>Coefficient of Variation (CV)</strong><span style="font-weight: 400;"> provides a robust method for assessing consistency across WiseWorld's dynamic episode scenarios. Given the platform's varied interactive episodes—each with different contexts, challenges, and targeted soft skills—CV’s adaptability is particularly suitable for an interactive, user-driven framework.</span></h4>
<h4><span style="font-weight: 400;">Key advantages include:</span></h4>
<ul>
<li style="font-weight: 400;" aria-level="1">
<h4><strong>Standardization across episode scenarios:</strong><span style="font-weight: 400;"> CV measures variability relative to the mean, enabling meaningful comparisons across diverse episodes and skill categories without bias from absolute score differences.</span></h4>
</li>
<li style="font-weight: 400;" aria-level="1">
<h4><strong>Dimensionless Nature:</strong><span style="font-weight: 400;"> The CV, a standardized measure, ensures fair comparisons across datasets of varying scales, making it ideal for assessing consistency.</span></h4>
</li>
<li style="font-weight: 400;" aria-level="1">
<h4><strong>Sensitivity to Data Distribution:</strong><span style="font-weight: 400;"> CV effectively captures relative dispersion even when data distributions deviate from normality, which is common in user-driven platforms like WiseWorld.</span></h4>
</li>
</ul>
<h4><span style="font-weight: 400;">By employing CV, the study ensures that reliability assessments remain </span><strong>consistent</strong><span style="font-weight: 400;"> and </span><strong>comparable</strong><span style="font-weight: 400;"> across various episode scenarios, providing a precise, objective measure of the platform's performance.</span></h4>
<h4><strong>Data Collection Tools</strong></h4>
<p><span style="font-weight: 400;">WiseWorld evaluates </span><strong>44 soft skills</strong><span style="font-weight: 400;"> through user interactions within its gamified episode scenarios. Scores for each skill are calculated based on two primary factors:</span></p>
<ul>
<li style="font-weight: 400;" aria-level="1"><strong>Alignment with the Target Soft Skill:</strong><span style="font-weight: 400;"> For example, a decision demonstrating leadership aligns with the episode's scenarios requiring delegation skills.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Contextual Fit:</strong><span style="font-weight: 400;"> Responses are assessed based on how well they integrate into the ongoing narrative and contribute meaningfully to resolving the presented challenge. This includes evaluating the relevance of the response to the scenario's specific context and the logical progression of actions taken by the user within the story.</span></li>
</ul>
<p><span style="font-weight: 400;">Scores range from -5 to +5, where:</span></p>
<ul>
<li style="font-weight: 400;" aria-level="1"><strong>Positive scores</strong><span style="font-weight: 400;"> signify successful engagement with the target skill, reflecting actions that effectively contribute to overcoming the challenge.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Negative scores</strong><span style="font-weight: 400;"> reflect missed opportunities or actions that detract from resolving the challenge, indicating areas where the soft skill was not effectively demonstrated.</span></li>
</ul>
<p><strong>Interaction Quality and Frequency: </strong><span style="font-weight: 400;">In the WiseWorld version V0, users engaged in chat-based interactions with an AI narrator (akin to a D&D game master), with each episode allowing up to five responses. High interaction was thus defined by users who reached the maximum of five responses, effectively completing the episode’s dialogue. This level of engagement provided richer data for skill evaluation. Notably, 57% of participants met this five-response threshold, indicating robust user involvement. Conversely, those exiting the episode contributed fewer data points, reducing the accuracy of their soft skill assessment. Ensuring a minimum number of meaningful interactions remains essential for maintaining the reliability and validity of evaluation outcomes.</span></p>
<p><span style="font-weight: 400;">With the CV methodology established, the following section explores the reliability of WiseWorld’s assessments based on this metric.</span></p>Results: Coefficient of Variation (CV) Analysis
<h5><strong>Descriptive Statistics and Reliability</strong></h5>
<h5><span style="font-weight: 400;">The Coefficient of Variation (CV) analysis across participants yielded a mean of 82.63 and a median of 89.00, with most users falling within a moderate variability range of 70–100. This clustering around the mean is a strong indicator of measurement reliability, as it suggests that individual assessments deviate only slightly from the average. This supports WiseWorld’s consistent performance across diverse episode scenarios.</span></h5>
<ul>
<li style="font-weight: 400;" aria-level="1"><strong>Mean CV:</strong><span style="font-weight: 400;"> 82.63</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Median CV:</strong><span style="font-weight: 400;"> 89.00</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Standard Deviation:</strong><span style="font-weight: 400;"> 24.83</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Minimum CV:</strong><span style="font-weight: 400;"> 26.37</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Maximum CV:</strong><span style="font-weight: 400;"> 112.25</span></li>
</ul>
<h5><strong>Supporting Evidence of Reliability</strong></h5>
<h5><strong>Consistency Across Diverse Populations:</strong></h5>
<h5><span style="font-weight: 400;">Most users—spanning various industries and backgrounds—maintained CVs within the moderate range. This consistency implies that WiseWorld’s AI-driven evaluations perform stably regardless of user differences, a key quality in psychometric assessments.</span></h5>
<h5><strong>Comparison to Industry Benchmarks:</strong></h5>
<h5><span style="font-weight: 400;">In many simulation-based assessments, higher variability is common due to uncontrolled narrative elements or less adaptive feedback systems. WiseWorld’s tighter variability suggests that its scenario design and adaptive mechanisms effectively stabilize outcomes.</span></h5>
<h5><strong>Theoretical Support:</strong></h5>
<h5><span style="font-weight: 400;">Research in assessment theory posits that moderate variability, especially with scores clustering near the mean, reflects a well-calibrated tool. This clustering indicates that scenario challenges are appropriately balanced, so results more accurately reflect accurate skill levels rather than external noise.</span></h5>
<h5> </h5>
<img src="https://wiseworld.blob.core.windows.net/blog-section/Visualizing_Variability_of_Users_Interaction_on_WIseWorld.webp"" alt="Visualizing_Variability_of_Users_Interaction_on_WIseWorld" width="auto" height="300" loading="lazy">
<h5><strong>Left Histogram (66 Users):</strong></h5>
<h5><span style="font-weight: 400;">The distribution for all users, including those with limited interactions, shows most CVs clustering in the moderate range, visually reinforcing the platform's overall consistency.</span></h5>
<h5><strong>Right Histogram (52 Users):</strong></h5>
<h5><span style="font-weight: 400;">Focusing on users with sufficient data (≥2 episodes) sharpens this picture. The histogram excludes those with fewer interactions, providing a clearer view of assessment reliability among active users.</span></h5>
<h5><span style="font-weight: 400;">Both histograms demonstrate that the majority of users exhibit moderate variability (CV: 70-100), while a minority of high-variability outliers (CV > 100) highlight areas for scenario refinement and adaptive difficulty adjustments.</span></h5>
<h5><span style="font-weight: 400;">The evidence strongly supports that WiseWorld delivers reliable, consistent results across diverse participants and episode scenarios by combining statistical clustering, theoretical underpinnings, and visual data from the histograms. The moderate CV range confirms that the platform effectively balances sensitivity to individual differences with overall stability, reinforcing its utility as a robust soft skills assessment tool.</span></h5><p><span style="font-weight: 400;"><strong>Outlier Analysis</strong></span></p>
<p><span style="font-weight: 400;">Approximately 15.4% of users displayed high variability (CV > 100), suggesting inconsistent decision-making or limited familiarity with the narrative roleplay-inspired format. In response, one of the updates implemented in WiseWorld version V1 moved from a single-narrator model to multiple intelligent NPC interactions, each with distinct personalities and adaptive behaviors. This shift reduces scenario ambiguity, allows for a broader range of user actions, and stabilizes the assessment process, ensuring a more accurate capture of typical and outlier user profiles.</span></p>
<p><strong>User Segmentation</strong></p>
<p><span style="font-weight: 400;">Participants were categorized into three groups based on CV values:</span></p>
<ul>
<li style="font-weight: 400;" aria-level="1">
<p><strong>Low Variability (<70):</strong><span style="font-weight: 400;"> 12 users (23.1%)</span></p>
</li>
<li style="font-weight: 400;" aria-level="1">
<p><strong>Moderate Variability (70-100):</strong><span style="font-weight: 400;"> 32 users (61.5%)</span></p>
</li>
<li style="font-weight: 400;" aria-level="1">
<p><strong>High Variability (>100):</strong><span style="font-weight: 400;"> 8 users (15.4%)</span></p>
</li>
</ul>
<p><span style="font-weight: 400;">Low-variability users demonstrated stable and reliable results, while moderate-variability users validated WiseWorld’s ability to accommodate diverse behaviors. High-variability users highlighted areas for refinement.</span></p> Discussion
<h5><strong>User Engagement Metrics</strong></h5>
<h5><span style="font-weight: 400;">The results highlight WiseWorld’s potential as a practical, scalable tool for organizations focused on soft skills. High engagement metrics indicate strong user interaction, which supports more accurate skill assessments. Planned enhancements—like adaptive episode scenarios and dynamic dialogues—will further strengthen WiseWorld’s capacity to deliver meaningful, data-driven insights for diverse groups.</span></h5>
<ul>
<li style="font-weight: 400;" aria-level="1">
<h5><strong>Episode Duration:</strong><span style="font-weight: 400;"> Each episode lasted approximately 7 minutes, aligning with industry standards for effective engagement.</span></h5>
</li>
<li style="font-weight: 400;" aria-level="1">
<h5><strong>Daily Engagement:</strong><span style="font-weight: 400;"> Users averaged 19 minutes of playtime daily, completing about three daily episodes.</span></h5>
</li>
<li style="font-weight: 400;" aria-level="1">
<h5><strong>Total Playtime:</strong><span style="font-weight: 400;"> Participants who finished all intended challenges averaged about 63 minutes of cumulative playtime.</span></h5>
</li>
</ul>
<p><span style="font-weight: 400;">Gamified assessments may favor users already familiar with gaming environments (Altomari et al., 2023). To mitigate this, WiseWorld includes a structured onboarding process that orients newcomers to its interactive elements, helping ensure fair assessment for all.</span></p>
<h5><strong>Addressing Potential Biases</strong></h5>
<p><span style="font-weight: 400;">Participants from tech-centric industries may be more familiar with AI tools, potentially influencing results. Tailored onboarding processes can mitigate these biases, ensuring equitable user experiences across diverse backgrounds.</span></p>
<p><span style="font-weight: 400;">Understanding these potential biases provides essential context for placing WiseWorld’s engagement metrics within industry standards.</span></p>
<h5><strong>Comparisons to Benchmarks and Engagement Analysis</strong></h5>
<h5><strong>WiseWorld’s Performance Against Industry Standards:</strong></h5>
<h5><span style="font-weight: 400;">Research in gamified learning environments typically reports average session</span> <span style="font-weight: 400;">durations ranging from 6 to 8 minutes (</span><a class="no-underline text-accent" href="https://www.researchgate.net/publication/256743509_Does_Gamification_Work_-_A_Literature_Review_of_Empirical_Studies_on_Gamification"><span style="font-weight: 400;">Hamari, Koivisto,</span> <span style="font-weight: 400;">& Sarsa, 2014</span></a><span style="font-weight: 400;">). In contrast,</span> <span style="font-weight: 400;">WiseWorld records an average daily playtime of 19 minutes.</span> <strong>This demonstrates</strong> <strong>high immersion and sustained value across multiple episodes, showcasing</strong> <strong>WiseWorld's ability to consistently engage users at a level that surpasses industry norms.</strong> <span style="font-weight: 400;">Given that users complete approximately</span> <strong>2.8 episodes per day</strong><span style="font-weight: 400;">—each</span> <span style="font-weight: 400;">lasting around</span> <strong>7 minutes</strong><span style="font-weight: 400;">—this elevated daily engagement</span> <span style="font-weight: 400;">is notably higher than</span> <span style="font-weight: 400;">industry norms.</span></h5>
<h5><span style="font-weight: 400;">This substantial difference implies that users are more immersed and consistently engaged with WiseWorld’s interactive, story-based episodes. Compared to traditional assessments, WiseWorld’s gamified approach offers a unique combination of engagement and reliability. Altomari et al. (2023) provide insights into the feasibility of serious games for soft skills assessment, offering a point of comparison for WiseWorld’s approach. WiseWorld addresses some limitations identified in similar tools by incorporating dynamic NPC interactions and adaptive episode scenarios. </span></h5>
<h5><strong>Enhanced Engagement Metrics:</strong></h5>
<h5><span style="font-weight: 400;">WiseWorld further distinguishes itself with robust daily engagement statistics:</span></h5>
<ul>
<li style="font-weight: 400;" aria-level="1">
<h5><strong>Average Daily Playtime:</strong><span style="font-weight: 400;"> 19 minutes</span></h5>
</li>
<li style="font-weight: 400;" aria-level="1">
<h5><strong>Day-1 (D1) Retention Rate:</strong><span style="font-weight: 400;"> 63%</span></h5>
</li>
<li style="font-weight: 400;" aria-level="1">
<h5><strong>Day-3 (D3) Retention Rate:</strong><span style="font-weight: 400;"> 41%</span></h5>
</li>
</ul>
<h5><span style="font-weight: 400;">These metrics indicate that users consistently return to the platform and dedicate significant time to each session. As gamification literature highlights, high retention rates, and sustained daily usage are critical indicators of effective engagement strategies (</span><a href="https://www.researchgate.net/publication/230854710_From_Game_Design_Elements_to_Gamefulness_Defining_Gamification"><span style="font-weight: 400;">Deterding, Dixon, Khaled, & Nacke, 2011</span></a><span style="font-weight: 400;">; </span><a class="no-underline text-accent" href="https://www.researchgate.net/publication/256743509_Does_Gamification_Work_-_A_Literature_Review_of_Empirical_Studies_on_Gamification"><span style="font-weight: 400;">Hamari et al., 2014</span></a><span style="font-weight: 400;">). For clarity, the </span><strong>D1 retention rate</strong><span style="font-weight: 400;"> refers to the percentage of users returning to the platform one day after their initial session. In contrast, the </span><strong>D3 retention rate</strong><span style="font-weight: 400;"> measures the percentage of users returning three days after registration.</span></h5>
<p><span style="font-weight: 400;">The platform’s engagement metrics, including an average daily playtime of 19 minutes and high retention rates, demonstrate its ability to captivate users effectively. </span><strong>This aligns with research showing that gamified solutions can significantly enhance user engagement while providing rich datasets for skill evaluation (</strong><a class="no-underline text-accent" href="https://aclion.com/evaluate-candidates-soft-skills-in-a-fun-and-engaging-way/"><strong>AC Lion, 2023</strong></a><strong>).</strong></p>
<h5><strong>Implications for Reliability and Assessment:</strong></h5>
<h5><span style="font-weight: 400;">The extended session durations and frequent daily interactions contribute to deeper user engagement, which enhances the quality and reliability of soft skills assessments. Longer and more consistent interactions provide a </span><strong>richer dataset</strong><span style="font-weight: 400;">, allowing WiseWorld’s AI to observe a broader range of user behaviors across multiple episode scenarios. This comprehensive data collection reduces the impact of outliers and random fluctuations, strengthening the platform’s assessment of human-centric abilities.</span></h5><h5><strong>Future Exploration:</strong></h5>
<h5><span style="font-weight: 400;">Further analysis could investigate the relationship between </span><strong>episode duration</strong><span style="font-weight: 400;"> and </span><strong>score consistency</strong><span style="font-weight: 400;">. Understanding how sustained engagement correlates with assessment reliability can provide insights into optimizing episode length and content, potentially leading to even more </span><strong>compelling user experiences</strong><span style="font-weight: 400;"> and </span><strong>robust data collection</strong><span style="font-weight: 400;">.</span></h5>
<p><span style="font-weight: 400;">Beyond metrics and comparisons, the user experience is crucial in engagement and assessment accuracy. The following section evaluates the UX enhancements made to WiseWorld.</span></p>
<h5><strong>UX Evaluation</strong></h5>
<p><span style="font-weight: 400;">WiseWorld’s design, inspired by role-playing frameworks like Dungeons & Dragons (D&D), </span><strong>Journey and Gamification Method:</strong></p>
<p><span style="font-weight: 400;">WiseWorld’s design draws inspiration from role-playing frameworks like </span><strong>Dungeons &</strong> <strong>Dragons (D&D)</strong><span style="font-weight: 400;">. It utilizes a gamified approach that immerses users in an interactive, decision-based journey. WiseWorld evaluates soft skills and transforms the assessment process into an engaging narrative adventure by creating episode scenarios that require deep engagement and thoughtful decision-making. This journey-based gamification method leverages familiar role-playing elements—such as character creation, story progression, and strategic choices—to make the process enjoyable while gathering meaningful data on user behavior and skill application.</span></p>
<p><strong>Observations and Limitations:</strong></p>
<p><span style="font-weight: 400;">The platform employs a single AI narrator to guide users through episode scenarios and collect their responses in the current iteration. Data revealed that over </span><strong>60% of users reached the chat interaction limit</strong><span style="font-weight: 400;"> with this AI narrator, indicating high engagement but also highlighting a potential bottleneck in interaction complexity and realism. This limitation suggests that a single-narrator model may not fully capture the nuances of dynamic human conversation or provide sufficient variability in feedback.</span></p>
<p><strong>New Approach to Enhance UX:</strong></p>
<p><span style="font-weight: 400;">To address these limitations and further enhance user experience, the updated version of WiseWorld introduces </span><strong>dynamic dialogues</strong><span style="font-weight: 400;"> with up to </span><strong>three smart Non-Player Characters (NPCs)</strong><span style="font-weight: 400;">, each possessing distinct attitudes and personalities. This evolution from a single narrator to multiple NPCs aims to:</span></p>
<ul>
<li style="font-weight: 400;" aria-level="1"><strong>Simulate Real-World Interactions:</strong><span style="font-weight: 400;"> By engaging with multiple characters, users experience a more complex and realistic social environment, mirroring real-life conversations and decision-making contexts.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Enrich Narrative Complexity:</strong><span style="font-weight: 400;"> The presence of multiple NPCs allows for branching dialogues and more varied episode scenarios, increasing the depth and adaptability of each interaction.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Improve Assessment Precision:</strong><span style="font-weight: 400;"> A more nuanced interaction model allows the AI to better evaluate soft skills across different conversation dynamics, leading to richer data and more accurate assessments.</span></li>
</ul>
<img src="https://wiseworld.blob.core.windows.net/blog-section/wiseworld_app_v1.webp" alt="wiseworld_app_v1" class="rounded" width="auto" height="300" loading="lazy">
<p><strong>Impact on User Experience and Assessment:</strong></p>
<p><span style="font-weight: 400;">While the current study validates the reliability and practicality of the single-narrator version, these updates are designed to:</span></p>
<ul>
<li style="font-weight: 400;" aria-level="1"><strong>Sustain user immersion:</strong><span style="font-weight: 400;"> Offering varied NPC interactions enhances meaningful participation and reduces the likelihood of repetitive dialogue experiences.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Increase Accuracy:</strong><span style="font-weight: 400;"> Simulating a broader range of conversational styles and episode scenarios provides a more comprehensive assessment environment.</span></li>
<li style="font-weight: 400;" aria-level="1"><strong>Elevate Gamified Journey:</strong><span style="font-weight: 400;"> The enhanced NPC framework deepens the narrative experience, making the journey more immersive and reflective of real-world social dynamics.</span></li>
</ul>
<p><span style="font-weight: 400;">By evolving the user experience from a singular narrative path to a multi-character, dynamic dialogue system, WiseWorld reinforces its commitment to </span><strong>engaging gamification</strong><span style="font-weight: 400;"> and </span><strong>rigorous soft skills assessment</strong><span style="font-weight: 400;">. This new approach addresses previous limitations and sets the stage for more effective and realistic evaluations, ultimately benefiting users and organizations.</span></p> Conclusion
<p><span style="font-weight: 400;">This study confirms that WiseWorld is a reliable and engaging tool for soft skills assessment, demonstrating moderate score variability and robust user engagement metrics. By refining its episode's scenarios and transitioning to multi-NPC dialogues, WiseWorld enhances assessment precision and user experience. As the demand for soft skills grows in an AI-driven future, WiseWorld bridges the gap between skill development and workforce readiness, offering a scalable solution for organizations worldwide. Future iterations of WiseWorld can further revolutionize soft skills development, setting a new standard for workforce preparedness in an AI-driven era.</span></p>