SIT-designed AI, VR tools used to assess workers’ skills

SINGAPORE – Artificial intelligence and virtual reality (VR) have transformed the traditional methods used to evaluate the skills of volunteers for social service agency Loving Heart.

The organisation needed a more efficient way to gauge how effectively its volunteers could spot hazards in the homes of its elderly clients, show empathy, communicate and make decisions about how best to engage with the seniors.

Traditionally, this evaluation was carried out during on-the-job training. However, this approach required assessing each volunteer individually, making trips to the homes of the elderly, and observing their interactions first-hand.

The new AI- and VR-powered test reduces the time and manpower needed for such evaluation, said Ms Elaine Ho, 47, a nurse clinician and head of community health at Loving Heart.

The social service agency worked with the Singapore Institute of Technology (SIT) to design the test, which is taken online.

Rolled out in January, it comprises authentic scenarios and tasks related to working with the elderly. Around 50 candidates in roles such as befrienders and medical escorts have since taken the test.

Initially, Ms Ho had doubts about whether soft skills could be tested using AI. But the test exceeded her expectations.

“How can AI test for empathy? I was not convinced initially, as we usually had to watch our volunteers interacting with elderly to assess this. But the interaction and evaluation by this AI tool blew me away,” she said.

Ms Ho added that the test results helped to identify volunteers’ existing skills attained through their own caregiving experiences caring for their loved ones. It also highlighted skill gaps, and guided the implementation of remedial training.

“This will allow us to scale to keep pace with the rising tide of problems of an ageing society, especially if we want to give personalised care,” said Ms Ho.

The online test is part of SIT’s Centralised Skills Assessment and Validation Initiative (CSAVI), a joint effort with SkillsFuture Singapore (SSG) to create a national standard for testing a variety of professional skills.

It grades and provides instant feedback for some components which do not have straightforward answers, but a human assessor will check and verify the final results.

The assessment comprises four parts. The first is a VR simulation of an elderly person’s home, where candidates need to identify safety hazards.

The next component involves interacting with an AI avatar designed to portray a disengaged and challenging elderly individual, who keeps saying the same things and expresses frustration. 

The last two components require typed responses: A short-answer test on care knowledge, and interaction with a chatbot programmed to mimic an elderly person for the assessment of candidates’ problem-solving skills and ability to pick up cues. 

SIT Associate Professor May Lim, who leads CSAVI, said the skills assessed in the online test were drawn from relevant skills and community care guidelines from SSG and the Agency for Integrated Care, which oversees aged care services in Singapore.

Upon completing the test, all candidates will receive a certificate – issued by CSAVI – indicating the skills they were tested on.

Ms Ho said: “(The certificate) benefits the volunteers, so that even when they move to other agencies, they won’t have to do the same training again. They can focus on other skills.”

Singapore Armed Forces warrant officer Yeo Joo Seng, 44, has been volunteering at Loving Heart as a befriender for more than two years. 

He took the online assessment in April and found it to be a useful learning tool, especially the VR simulation of an elderly person’s home.

“This kind of assessment is essential for equipping us with the skills to identify potential risks and hazards in our client’s house. Beyond just being an assessment, it is also a good guide and practice tool for us,” he said.

CSAVI is expanding its services and creating skills assessments for those in the hospitality and technology industries.

It is working with Resorts World Sentosa to create an AI-driven system for evaluating customer sensitivity, service excellence and operational agility. 

The assessment will also facilitate skills-based hiring, by identifying candidates with the right service-oriented traits, as well as highlight retraining needs for existing team members to ensure high service standards.

Prof Lim said the initiative is essential as Singapore shifts towards a workforce valued for what workers can do, rather than solely for their academic credentials.

“Assessment and skills validation are part of the puzzle for the whole skills-first economy to work,” she said.

While career growth has traditionally relied on self-perception – where individuals rate their own strengths – Prof Lim said there is a need to shift towards objective validation. 

She added that by using AI-driven assessments by CSAVI, workers might discover hidden talents that they never would have identified through self-rating alone.

“The test might say, ‘Did you know you are very good at data storytelling?’ You might not have known (that), but the test provides something more objective on top of your self-perception,” she said.

If a skills gap is identified through the assessment, Prof Lim said CSAVI will be able to guide candidates in their upskilling through Singapore’s entire training network, including courses at ITE, polytechnics and SSG.

Cragar Industries, a Singapore-based medical technology manufacturer, turned to CSAVI to help identify employees who can lead the company’s digital transformation. 

Mr Han Myo Thaw, general manager at Cragar, said small and medium-sized enterprises like theirs often struggle with training due to a limited budget.

“I can’t send people for training everywhere all the time. We need to be a little bit more focused so that we can tailor a learning experience that is actually effective,” he said.

Cragar has moved away from traditional multiple-choice tests and one-to-one interviews, opting instead for CSAVI’s interactive AI platform. 

Mr Han said this allows the company to assess staff on how they handle complex scenarios, like mediating workplace conflicts.

Describing the interactive platform as more of a conversation than a test, he said: “If a staff member provides an unsuitable answer, the AI would prompt them to rethink their answer or consider another solution.

“It is really not about the score… but also allowing the individuals to become aware of their own capabilities so they can beef up their own learning.”

Fifteen of Cragar’s employees have completed the assessment in three areas – AI essentials, digital literacy, and soft skills like resilience and problem-solving. 

Mr Han said the results provided “surprising data” that challenged management’s assumptions. He said: “We found people in executive roles were using AI tools more than some of the staff we thought were more proficient.”