Name: The Skills Reckoning: Why Inference Isn’t Enough
Uploaded: 2025-06-17T17:16:04.690Z
Duration: 1 h 12 s
Description: The Skills Reckoning: Why Inference Isn’t Enough

Transcript for "The Skills Reckoning: Why Inference Isn’t Enough": Hey. Good morning, everybody. Good afternoon. Appreciate you joining us today for, our chat. Topic that is not topic for sure. It's all this work about still is to talk about why the conversation, needs a bit of a reboot reboot and is broken. In terms of introductions, first, I'll introduce myself. I'm Mike Hudy. I'm a chief science officer at Hirevue. I've been doing this for, quite some time. Even though the skills conversation is one that's, really picked up steam over the last few years. I'm an industrial organizational psychologist. We've been talking about, skills for decades. So it is one that's near and dear to my heart, and I know it is to, our esteemed guest, Madeline Laurano. Madeline and I go back quite a ways, friends and colleagues. She's the founder of Aptitude Research. Madeline, I'll let you introduce yourself to our audience. Thank you, Mike. I'm excited to be here and talk about this. So we've been talking about the topic for a long time, so to see it be so popular and interesting now in this time that we're in in this market is very exciting. So, it's my honor to be here. I'm very excited to be here. I'm the founder of Aptitude Research. We do research on HR technology and talent acquisition technology. So we did this study and partnered with Hirevue on it to really look at the difference between, inferred skills and validated skills. And to your point, Mike, the conversation on skills is not a new one. I mean, we've been talking about this for a long time. Really started to gain some momentum over the past few years, and And then we started to notice that there was a lot of fatigue on the conversation on skills. A lot of companies felt very stuck. And we found that a lot of these companies were just really using inferred skills, not seeing a lot of traction, feeling a lot of frustration. So we decided to look at this from a very different angle, focus on validated skills, focus on assessments, and the value that it can really bring to this this whole skills conversation. So, I'm excited to talk today, and I'm excited to share some of this new research. Excellent. Thanks, Madeline. And I think, stuck stuck is a good word, and, I think we'll we'll we'll unpack that a little bit in terms of what we mean by stuck. And, hopefully, by the end of the webinar, today, we'll have some good practical things, for helping people and companies, get unstuck. Before we dive in and and can't wait to jump into I think the, survey results were were pretty fascinating fascinating, and and some of it's confirmation of what we already heard, but, we're seeing it in a big way from those that participate in the survey. But before we get into that, I did wanna ground us a little bit on the skills conversation as a jumping off point. So, obviously, if you've joined this webinar, you've probably been part of the skills conversation. There's also a pretty good likelihood that, you're suffering from some level of fatigue as well. And so the question first is is why skills? Why has there been such an explosion of discussion of skills? And for the purposes of today, we're really gonna hone in on, skills based hiring. There's really good reason for it, and lots of different trends that have led us in this direction. But the biggest one is the massive change that we're seeing and continuing to see with jobs. And the biggest disruptor lots of disruptors in the job market, but the biggest one is artificial intelligence, and the speed with which jobs are changing and evolving. The for instance, the World Economic Forum, suggests that over the next five years, a 170,000,000 new jobs will be created and 92,000,000 jobs will go away. So massive amount of change just in a five year window. With that kind of change, what they also quote is that 39% of the skills required for jobs are also gonna change. And then with that not surprising, 63 of employers are saying that they're running into skill shortages. So how does that get us to skills, skills based hiring? There's been an acknowledgment that with the massive change and knowing that jobs are so fluid now, do new jobs being created, current jobs changing rapidly, the kind of historic practice of looking at a resume and looking for past experiences. I need to hire somebody in this role, and I need to find somebody with five years experience who have done the same exact thing. That model breaks down when everything is changing, and in five years, we're having this massive amount of change. So it's kinda led us to this idea of, like, well, instead of looking at like for like past experiences, we need to break jobs down. We need to break jobs down into their component parts and start looking at transferable skills. Not that somebody have this experience, but rather do they have this skill regardless of whether they got that skill. So conceptually, it's great. It makes a lot of sense. But why we're here today is because of this fatigue, because of the stuck in this that Madeline talked about, And that is because on paper, very easy conceptually sounds great. But then when you actually start to put it into practice, that's where it becomes, that's where it becomes more challenging. So with that, want to kind of introduce and start to talk about some of the survey results as a jumping off point for our conversation. And, you know, as Madeline mentioned, we had an opportunity to partner, on this survey. And, one of the thing and, Madeline, I'll let you talk a little bit more about the survey since it was your survey, in terms of who was who was surveyed or, who responded to this survey. But, like, one of the key takeaways from that, and we've already mentioned, is that fifty percent of folks that participated in the survey say that they are suffering from fatigue around this conversation. And so kind of first question to tee things up, Madeline, is just why? What what's what's the driver of so many people, feeling this level of fatigue from the skills conversation? Yeah. It's really interesting, and I see it not just with, HR leaders, but also I just feel like the industry overall is feeling this fatigue around the the skills conversation. I think it's a few different reasons. I think one is a lot of companies take a very different approach to a skill strategy, to the technology that they're investing in, and then to the change management that goes along with that. So they're almost viewing it as this piecemeal approach to skills, and it takes a long time. The idea for anyone to kind of come, you know, to market or to kind of come and transform their company with a skill strategy, it feels like it's gonna take five years to do that. And I think a lot of companies get stuck with that reality. And they'll even go internally and talk to c level executives or even go to the CFO and say, you know, we wanna budget around skills. It's gonna be relevant in five years. That's a hard thing to push across right now. So I think the this stuck on skills, you know, one frustration is that it's gonna take too long, and a lot of companies are seeing that. You know, we even you know, we've attended conferences together, you know, in the past few years, and a lot of the presentations and a lot of the case studies around skills haven't really shown anything. It's just companies talking about what you were just talking about, why skills, why now. And that's important and we know that, but we have to move from the why to the how are we gonna do this. And a lot of companies don't have examples or have not yet moved to the how. So I think that's one reason. I think the other is there has been a heavy reliance on just inferred skills to be able to really execute on a skill strategy. And companies are looking at a resume, and they're inferring skills from a resume and making assumptions and, you know, kind of adopting a skills based approach based on those inferred skills. And that's important and it's great and I'm, you know, I'm a believer in inferred skills, but it doesn't really predict performance. It doesn't really provide a complete or accurate view of an individual, whether that's an external candidate, whether that's internal talent. It's not really, painting the full picture. It's not based on science. I know that's important to you, Mike. So there's a lot of concerns with this inferred skills based approach, which is where a lot of companies have landed, and there's been a lot of frustrations with it too. A lot of companies have said, okay. This is our approach, but if we look at our, you know, high potentials or whatever you wanna call these high performing individuals within your own organizations, a lot of companies test the model and say it doesn't really work on who we would define as quality hires once we've hired them. So we have questions about that. So, you know, I think with the idea that the skill strategy is gonna take too long and this idea that we don't necessarily always trust inferred skills, there's been a breakdown in moving the conversation forward. Yeah. Great points. And, I wanna pick up on that. But before I do, I had, like, two jobs in the introduction, and I failed on one of them. So I do want to let the audience know that we will have time at the end for questions, and you should see a q and a, in your chat. So, as you have questions, we won't address them necessarily in real time, but we do have, time, at the end, to dig into some of those. So, please do put your questions in the q and a. So picking up on that, and I love those points, and love the term stuck, and trying to to kind of, boil the ocean and kind of come up with the skills strategy. I I think another you know, when I think about this, and I've had a lot of conversations about what gets people stuck and also attend some of the same conferences and hear it, but actually go to, like, the nerd fest at SIOP and hear some of the same conversations. A big one where I think I see organizations getting stuck is around kind of defining the skills taxonomy. Like, the first step is we have to have this overarching skill strategy, and that that starts with a skills taxonomy and which one's perfect and which one are we gonna use and how are we gonna use this across all of our jobs. And that's fine. Like, it's good to have a lexicon, by which you can start to talk about that skill conversation. But where it can be challenging, and, again, where, we could fall down a little bit, in in terms of, what do taxonomies actually do for us from, again, applied to skills based hiring, they can only take us so far. And I think that's where, again, companies can get stuck, and this is just an example to kind of illustrate this point. So let's say we we're an organization. We have, we've put a skills taxonomy in to place. And here, I'm using one particular job and a call center example. We do a lot of work. I personally do a lot of work with call centers. So it was an easy example for for me to do. Here, I didn't have a taxonomy, so I did the next best thing. If you don't have a taxonomy, what do you do? You go to ChatGPT, and I just a simple prompt of, hey. I need to hire for a skill a a call center, associate. What are the top 20, skills of a of a call center associate? And I've like, off it often does. ChatGPT did a good job, returned 20 of these skills. And for those of you that know call centers, have call centers, have worked with call centers, you can look at this list. And there's nothing that you would necessarily disagree with. Like, yeah, I knew my call center rep needs to have empathy, communication skills. I need to be able to multitask, patient. I be patient. I can't argue with any of that. But it's still what it is is basically a laundry list of skills. So how now do I take this and turn it into action? How do I hire against it? How do I how does this support me? And if I'm going out there and looking for, candidates, do they have to have all 20 of these skills? If they don't have all 20 of them, is 15 out of 20 of these skills passing? So it doesn't give us actionable, where do I need to take this and how do I use it? You can dig a little bit deeper on that as well and say, well, here's 20 skills. Are they all created equal? Like, are they all equally important? Does somebody this multitasking just as important as time management in a call center. It's it's just a list. It's descriptive. It's a great starting point. But what it lacks is, okay. How do I turn this into action, and how I how do I actually start hiring against that? And, yeah, as Madeline said, science is pretty important to me personally, to our organization. So this is where okay. Taxonomy is great, defining what those skills are. But then where do you go with it? What's the road map for them? How do I hire against it as well? Skills now meet predictive science. Predictive science in this case is really just saying, how do I know which of those skills is more most important? Well, the way you know that is data and data about actual hiring outcomes. Like, which of these skills are associated with great quality of hire? All of this, the metrics in the in the contact center and people who stay. And the answer is, not everything on your laundry list tells you who the superstar is gonna be and who's the not so good one and who's gonna fail at the job. It's generally a subset of those. And we've done this kind of research, and we've done it with dozens and dozens of call centers as well as other organizations. But here, I'm using a call center example. So what this is is, first, it's just competency. So across these dozens of studies we've done, these are the core competencies that come up again and again and again in terms of driving success in the role drive for results, following rules and procedures, etcetera. So this is high level competencies. But beneath this, the building blocks of these competencies are skills. So let's dig into that. What does the data say? Well, the data says, if you're looking at produces quality work for somebody in a contact center role, it's not the it's not 20 different skills you need to index on. It's actually five. It's can they multitask, do they have computer skills, can they cope with pressure, results orientation, and achievement orientation. Those five things. If you know that, you're gonna have a pretty good idea of whether this candidate will be successful, specifically with producing quality work. Another example, produce, providing exceptional service. Service, really important. Don't need to measure everything. Don't need index on everything. There's actually our data says when we pull in data that says what drives quality of hire, it's customer service, orientation, perspective, taking desire to learn, etcetera. So this starts to guide us into we have this taxonomy. We have this idea of what the skills are, but what skills really matter from a hiring perspective. Furthermore, all these are are these created equal? Spoiler alert, they're not. I probably wouldn't answer asked the rhetorical question or not rhetorical question if that was the case. The ones that are starred are more important, and our data shows that, like, way more important is multitasking for producing quality work. So, like, if you can't multitask, but you have computer skills and you can cope with pressure, you're still probably not gonna produce quality work. From a customer service orientation, customer service orientation is three times more critical than the others on this list. So all of this is then, how do we take this laundry list of skills and turn it into action? It's like these specific things, here's how we weight them. And so when we have our science solution, our assessment solutions, and you're putting them into place, all of this is happening behind the scenes, and it's giving you, overall fit score. But behind the scenes, it has that predictive science to say, I'm only measuring what matters, and I'm weighting it in a way that links to these outcomes that I care about. So this starts to move us in the right direction in terms of how do I get from, thinking about skills, conceptually, they're great, to putting them into practice from hiring. But there's still more. Plenty of reasons where it can fall down. And Madeline kinda teed us up with one at the start. It's like getting into the measurement and this idea of, skill inference and, versus skill validation. So we're gonna dig into that. And, Madeline, in the survey, we did or you did ask, some around this and some some pretty telling, insights from this. But before I ask you to share some of that, what I do wanna do is just ground this conversation because we will use these terms and just wanna make sure the audience is familiar with them and what we're talking about. So there are different ways to measure skills, and none of them are inherently good or bad. We're not saying that just one's good or one's bad. They all serve a purpose. They're all important for us in this kind of quest to understand somebody's skills. What we've done is put this continuum together, on the three main categories of the way that's that you can measure skills, and we've organized it around degree of certainty. Meaning on the left, the most uncertain as you move to the right, the most certain in terms of do we really understand this person's skills. So just a little bit of color on each of these. So when we say inferred skills, and Madeline will talk about already mentioned it in survey results. When we say inferred skills, this largely comes from us from using artificial intelligence and artificial intelligence, mining many, many job descriptions and, resumes to say what skills kind of go together they correlate. If somebody says that they a candidate an individual says that they have skill a and b, they probably also have c, d, and f because we see that come up a lot in the data. So when using it from a hiring perspective, it's great for uncovering things that a candidate might not have told you. If they filled out an application or gave you a resume and they listed some skills or some experiences, the AI is able to say, like, hey. This person may also have c, d, and e. That's great. You've uncovered something. You've potentially uncovered something. We describe this as more speculative information that you need to go like, I need to go ask the candidate about that. I'm not gonna hire them based on I speculate or I've inferred that they have these skills. Well, hopefully, you're not. So that's inferred skills. Self reported skills are kind of the next level of certainty. Self report skills are those that candidates, individuals are telling you that they have. They're listing their previous experiences. They're calling out skills that they've developed from those. So a higher degree of certainty, they've actually said they possess these. They're descriptive of the individual. But, again, level of certainty, would you ever I don't think we have any customers that take the resume, review it, and hire a person because, like, man, this person has all the skills we're looking for. You're hired. When can you start? It's a jumping off point. It's a great description. And what happens after that, typically, we're gonna interview. We might give an assessment because we're looking to the last category to validate those skills. And we say validate those skills. What we're saying is more direct measurement of those skills. And those can be a combination of interview, assessment, where you're actually seeing those skills in action. It could be looking for certifications. So if you're hiring for a finance position, series 63, they've been certified to have this skill. So that's just kind of grounding. When we say inferred, self reported, invalidated, this is the terminology we're using in kind of the the classification scheme that we're using. So with that, Madeline, I know there were some good questions that you you asked in the survey. And I guess, first, why do so many out there still rely on inferred or self reported skills? Yeah. It's I love this slide because it's a building on, I think, where companies should go and what they're thinking about. And I like how you talk about it too because the alternative to inferred skills is you're just taking resume as it is, and there's a lot of bias in how people evaluate resumes. There's a lot of bias in even how people create their own resumes. So, you know, the alternative of inferring skills is a positive thing because it's better than, I think, what was happening before AI was involved, and we were just using a resume at face value or there was, you know, a lot of bias being, you know, taken into into account in that process. The self reported, it all is in some ways a better approach too because we haven't had input from candidates in a way that we need to, and we haven't had people say, okay. I actually, am, you know, skilled at this or I actually do have these skills, and I haven't had the opportunity to be able to really showcase that. So there there's positives to those, but what they don't do is valid they're not validated and they're not based on science, and they're not helping to inform decision making at the end of the day. And that's what companies really need. I think we've had such a reliance on, AI inference and and thinking about inferred skills because there are so many providers out there and there are so many examples. There are so many, job matching providers out there, and it's a really hot market right now. There's no shortage of providers that are part of larger suites and larger ATS systems. All the ATSs do this, and there's very, niche, very specific providers that just do this. And there's a demand for it, so more and more providers keep emerging. Not all providers are the same. I think that's been a very tricky case too. A lot of providers take a very responsible approach to AI. They're very transparent with the matching and how the inferred skills, are being used, but not all providers take that same approach. So we've seen this sort of crazy market emerge around this inferring of skills, and it's not been apples to apples. It's not always been responsible AI. So there has been a lot of, you know, a a lack of trust, I think, for a lot of this this area. But it's, you know, I think for a lot of companies, they also view it as efficiency. It's a it's a time saver. And if we think about recruiters and how they have to go through a large number of applicants, this can help save time. And there's a lot of value in saving time. There's a lot of value there. But we also need to make smarter science based decisions around talent, and that does not happen necessarily with how companies are are using, AI or thinking about skills in this way. So what we found is that 72% of TA leaders are only using AI inference for their skills based strategy. That's what they're calling their skills based strategy. And you can see here there's, a lot of distrust or missed feelings of of trust here around that process, I think, for the reasons that that I stated earlier. Yeah. I think I I love this slide or I Oh, I think we lost your volume or I'm not hearing it at this time. I think we may have lost it, but, I I I know I know the reason that you do like this slide too because I think it shows, that there is a lot of, you know, distrust around AI inference, and it's it's hard to know how companies feel about that when there's been such a demand and such a heavy investment in in this area of talent acquisition so, you know, early in the process. Just checking if you can hear me now? Yes. Okay. Thanks for saying what I would have said in the slide. Yeah. But I what what's I said, I guess, to myself only, is that kind of the three quarters of people use Inference only as their skill strategy late next to, but only 12.1% completely trust it. So I think that that really kinda nails the challenge that we're seeing and how do we actually implement and go about using the skills based strategy. So kinda next question to you is, this is saying that these are the strategies we're using today. We don't fully trust them. What do we do to start moving the needle of trust? Like, what did we learn from people in terms of what we can do to to start to build more trust in how we're going about measuring skills? Yeah. And I think the interesting thing is that companies don't wanna go back. I mean, the alternative is going back to recruiters have to manually go through every single resume. They don't wanna go through that process. We'd all probably love for the resume to be dead. That's not gonna happen tomorrow or probably in the next few months, But there has to be this in combination with what, you know, you showed on that slide earlier with the validated skills approach because, that's what will build the trust in, you know, how companies are using inference to support their skills based strategy earlier in the process. Yep. And then we found on this slide what would you need to change to improve that trust. And you can see here, it's it's a mix of a few different things. So the two things that stand out, 29% of companies said a combination of AI and direct skill validation. You you need absolutely need both. The AI is bringing an efficiency gain, it's helping companies, you know, through that first step in the process, but we need a direct skill validation as well. And then the other piece is better integration with existing HR systems, and I think this was really interesting to see as well because, you know, that is a challenge for companies. I mean, so many companies during the pandemic just bought and bought and bought and bought, and they invested in so many different solutions and so many different providers, and there wasn't a lot of integration. And when you talked earlier about the taxonomy challenge with the stuck on skills, there are providers that do skills taxonomy, there are providers that do skills inventory, there are providers that do skills ontology, and it is just too much. We have to think about what is, you know, an integrated approach to all of this. So, you know, it's inter it's interesting to see this here. I mean, also, 18% want more transparency and AI driven recommendations. This was something that you know, I've just gone through a series of demos with larger, ATS and CRM providers recently, and the transparency piece was something we've been talking about for a long time. I know, Mike, this is super passion for you. But not all providers were doing this, and it almost didn't feel like they had to a few years ago. It was, you know, something we talked about, but companies would not really ask. We didn't really expect to see it on a demo. And now it is a nonnegotiable. Like, you have to provide the explainability. You have to provide the transparency, and, you know, it's it's absolutely critical. So there's a lot of providers that do it and have made changes to be able to provide that transparency, and there's a lot that do not still. Yeah. And I was actually, gonna call that one out as well, in that, you know, this you know, there's this lack of trust. And a lot of times, what goes hand in hand with the lack of trust, especially when using technology solutions and AI solutions, is a lack of transparency because people don't know what's going on inside the black box, and it's hard to trust when you get recommended candidates. You know, there's an algorithm and AI is doing its fancy stuff behind the scenes, but you don't know what it's doing. And I just think it is more of an expectation now. It's more of an expectation, but it's also that, the state of the science and artificial intelligence continues to evolve, which is enabling, providers like HireVue as well to, provide more transparency. We all see what happens, the power of large language models and being able to use those to to provide a more a little bit more color around, and this is why this person is being recommended or here's the specifics around how this and why this person matches starts to build a little bit more trust. One more that I'll call out here, that makes me a little bit sad, is that, like, only 13% cite proof. And I just think in HR and talent acquisition, this is a a a challenge with the field in general. It's like we don't demand enough proof. We're putting all these solutions in. There's all these technology solutions. And ultimately, what are they there for? They're there to help you hire faster. It's generally easy to show that, but also to get the right hire. That takes a little bit more work, but it's why we hire in the first place. It's to get the right person. But only 13% are kind of quoting that, like, hey. I need a little bit more proof, and I I think it's huge, like, to the extent and I'll come back to this point, a little bit later. Like, coming back and saying, hey. We've shifted the skills based hiring. Here's some of what we're seeing, and it's not just a speed story. It's like, hey. We're we're filling roles now. We're doing it faster. And look at the quality as a very powerful and probably the most powerful, because ultimately, again, it's about getting it getting it right. Yeah. You're absolutely right. I mean, as an industry, we don't look at outcomes the way other parts of the business do. I mean, everything in other parts of the business are outcomes focused, and and we haven't been in this industry. And for a lot of companies, it's a struggle now because there's more CFO scrutiny, there's more business scrutiny, there's more, you know, eyes on kind of what we're doing where, you know, I think during the pandemic where everyone was scrambling, there was, you know, a lot of hiring happening in a very short period of time. There wasn't the same amount of scrutiny. Now it's the the outcomes piece is a huge mess for companies that are not doing it. And providers have responsibility in helping to support that outcomes conversation too. It's not just all on the company. Yeah. No. It's a great great call out, and it's kind of a misnomer with kind of HR and TA as the rest of the business is held accountable for moving the needle on metrics. But, that doesn't happen as often in HR and TA. Or if it's metrics, they're they're more kind of efficiency metrics, as opposed to, metrics about, what are you getting to give us the talent that have these skills that are gonna help us, realize, our company strategy? So to kinda transition us into into the next area, we've kinda talked about the lack of trust and and why, and the need to get beyond inference and self report to validated skills. And so, talk a little bit about what you're seeing or what we're seeing today in terms of how organizations are going when they're doing skills validation, what they're doing to validate skills. Yeah. So, so I the the first call I'll say is that only 12% are very effective at validating skills, and I think that says something there's such an investment on just inferring skills. We haven't really gotten to the heart of this conversation, which is the validating skills piece. So that was surprising and, you know, disheartening to see. And then I'll call it the other piece that was disheartening was the 23% reference checks. I don't know if that stood out to you first, but it did to me because that is not gonna validate skills. We all have been through that process. We've all given references when we've gone through the interview process and we've applied for different jobs. We've all probably had to check references for different hires that we've made or our organizations have made, and we know what that process entails. It's just to check the box. There usually isn't you know, it's it's come a long way with some technology providers doing more here, but to see that so high, was a little disheartening. I do like to see 35 said direct skill validation. That's what we're talking about now. I think that's why this report has been so popular and this discussion has been so popular is because there has been this I feel that there's really been this resurgence in the direct skill validation conversation through assessments and through, you know, structured interviews. And that's there's been a demand for that. I think companies see the value in that. That is an out based approach outcomes based approach. So to see that, that was, you know, what we'd hope to see. Yeah. So, one, that reference checks are on there and then just shocking. I mean, not that people don't need to do reference checks, but in the response to, what do you do to validate skills, and then to see. It's the second biggest slice in the pie, was was surprising, for sure. So I do wanna move more into the into the conversation about okay. So how do we go about validating skills, and to kinda tee this up, wanted to share a quote, from one of our customers, Laura, and I don't know if, Laura Fields is on today. But, Laura has a lot of good quotes, and, this was one, recently that kind of underscores a lot of what Madeline and I have been talking about is that there is a there's and I I won't read it. You can certainly read it. But to summarize it is there's tons of research out there that says past performance has very little predictive power in terms of future performance. There's also tons of research on, structured interviews, assessments, job samples that do have strong predictive power. But what's happened is we still, as a field, hang on to that resume, but technology has made, direct skill validation more accessible, more scalable at this time. And so, there really is an opportunity here to move, and only 35% doing direct skill validation, there really is an opportunity for us to make that move. We don't have it here on the slide, but the other great quote from Laurano as well is, as soon as you open a resume, you're no longer doing skills based hiring, which kind of, is is kind of the follow-up, I guess, to to this information. So let's talk about how do we move towards skill validation, what does it look like, and, also, we've already touched on artificial intelligence quite a bit here, but wanna talk a little bit about the role of artificial intelligence in validating skills. And, actually, I'll kind of start us there, in kind of my perspective on, artificial intelligence and validating skills. So first of all, so we're all we're all probably using artificial intelligence, super powerful tool. It's changing the way that we do our jobs. It's changing the way that we hire. But when we pull back and say, okay. What is artificial intelligence? Artificial intelligence is really it's just a tool. It's powerful, quote, unquote, math. It helps us uncover relationships and data that we otherwise wouldn't see, but it is only a tool, and it is only as good as the data that it sits on, the data that it's trained on. So I'm sure most of us today use some kind of large language model day to day, in their jobs, whether it be chat, GPT, Gemini, or some other large language model. And you know that you've been trained or you could, like, take the results that come back with a grain of salt. And why is that? Is it doesn't claim to necessarily be a 100% accurate because of the data that it's trained on, the data that it sits upon. It's like the Internet as a whole, and we know that the Internet there's reliable information, there's less reliable information. So, when it's coming back with, with answers to our questions, they may be on target, but they may not be. If we take that out and we apply it to the skills conversation, so most of what's driven the skills conversation or this big wave where everybody's jumping in has been technology and artificial intelligence. And taking artificial intelligence, and developing skills taxonomies and ontologies and how that was done and kind of the data that that sits upon is largely job descriptions and resumes. So each of those. So first job descriptions. So there's not a week that goes by in my role at Hirevue that we ask companies for a job description when we're working with a certain job. And before they send it, they always caveat it with, but our job descriptions are garbage. Our job descriptions aren't very good. I hear it every single week. Now there's probably companies out there that have fantastic job descriptions, but there's a whole lot of them that don't have very good job description. What does job good not good job descriptions mean? It can be a host of things, but the heart of it is, is it really a good description of the skills that you need to to do this job? So that's one source of information that these models are built upon. The other is the resume. We've already touched on some of the challenges with the resume, and I'll highlight some of those. So first of all, it is it's nonstandardized, so there's no kind of structure in the way that candidates go up put these together. There's there's loose rules that apply, but it's kind of a free for all with lots of tools out there to help candidates put these together. There always have been, and now there are more powerful tools. Again, with large language models, it's very easy, to just put a prompt in and and have, ChatCPT put together an amazing, resume inspired by the truth. And there's also a lot of research out there that documents how often, candidates actually embellish or just flat out why in their resume. So the two big sources of information that these models are built on, I'm not gonna say they're complete garbage, but they're they're decent at best, and there's questions with those job descriptions or resume. So not to say you have to throw out your taxonomies or your ontologies, but the role of AI invalidating skills gets to let's consider what we're applying the artificial intelligence to. And we can use it on resumes to infer skills. We can get self report skills. But if we actually wanna make a hiring decision, we need to go beyond that, and that's where, we get into direct measurement of skills. And I I talked a little bit about some of these methodologies already. Some were cited in Madeline's, survey results that she shared, but these are examples of more direct measurement of skills. On the far left, you have a coding challenge. So instead of looking at a resume and seeing if a candidate possesses, or having AI look at a resume and see if they possess, Python coding skills, I'm gonna feel much better about giving them a coding challenge on Python and seeing how well they can code. In the next one to the right, that's a multitasking simulation. In the multitasking simulation, I'm hiring from a call center again. I need someone that can multitask. Trying to pull that out of a resume at AI or not is very difficult, based on past experience. I really don't need to care about past experience. What I wanna know is that they have this skill. This skill is being able to split split their attention between multiple competing tasks. That typically is not gonna show up on a resume, but you can measure it directly and you can measure it really well. Next one over is an interview. Most of you, I'm sure, do interviews, and they are a way of doing skills validation, especially if they're done right. Done right, structured behavioral. So let's say we're tapping into we wanna hire a leader, so I ask leadership, interview questions. Tell me about a time when you led a team that you needed to turn around. What did you do? How successful for you? What did you learn? You're pulling out skills, of that individual through doing that interview. And, obviously, technology and artificial intelligence, can be a huge help there in, in pulling those skills out of, interview responses from candidates. And then the far right is a language, proficiency test. So, again, I need somebody that can speak fluent Spanish, hard to get from past experience or the resume, but, much more powerful to measure it directly and use artificial intelligence to do that. So, again, it's measuring it directly, and then AI becomes an even more powerful tool because it's laid against this data of direct measurement of, skills. Natalie, I know not necessarily in this survey, the past surveys, you've also looked at kind of HR and HR adoption of artificial intelligence and kind of share your thoughts and what you were seeing, with how HR is using, and adopting AI. Yeah. It's definitely increased. And I think the the biggest shift that I've seen this year is there was a lot of hesitancy last year. Right? There was a lot of hesitancy around regulations, around maybe bias, around, you know, work councils that companies had, and everyone was sort of moving very cautiously. A lot of even vendors weren't using AI at all because they were a little bit nervous about, you know, the interest or the appetite around AI. It seems to have shifted completely overnight. It's all of a sudden there's a green light for every company to invest in AI. They see the value. They've been given even budgets or increased budgets to invest, but there's still a lot of confusion about what companies understand AI to be or how it's being used or even what providers have AI. So I think there's been increased demand and interest in AI, which is exciting. I mean, I'm a huge believer in AI, responsible AI, and I'd love to see that. But at the same time, there's so much confusion out there, around, you know, I think, companies' understanding of providers and then what AI is actually doing. We see here you can see it's a real mix of how companies are thinking about AI and using AI. But the the piece that I I think is great to see is that 33% of companies believe AI has the potential to make the process more human. We'd like to see that higher, obviously, but, there's you know, obviously, that's a big difference than what we saw a few years ago. We actually survey candidates and ask them similar questions around, you know, would they be okay with AI or how what are their feelings around AI? And the percentage for candidates is actually higher for believing that AI has the potential to make the process more human. So I think overall between candidates and then HR departments and TA departments, there has been, you know, an increased interest in in AI and the potential that it has. That's interesting. I think we've seen the the shift as well, and it's been, it's been pretty dramatic just in the last year, eighteen months. I and it started with we did a we did an AI survey survey recent fairly recently within the last year, and we did see way more adoption from, users, from, talent acquisition and using it. But the other thing that changed, quite dramatically, which was encouraging, is the perceived fairness, from a candidate perspective. There was, and I think the more and more, transparency there is and the more cuss like, fewer examples of AI gone wrong that helps get by and use and it starts to move from this is something that's going to prevent me from getting a job to something this is something that can actually help me highlight my, highlight my skills. From a customer perspective, I think I knew, like, the tide had definitely changed when some of the most regulated organizations that we work with, like big banks, started this went from, like the the narrative used to be, like, we'll probably never be able to use AI to, like, all of a sudden it's, like, approved. We gotta get it into our ecosystem now. So and that's just happened, like, over the last twelve months. So to kinda transition us into, q and a, I do want to and, just a reminder to put questions in the q and a if you haven't already. We'll get to those here in just a second. But just to kind of put a bow on this, before we get into the q and a, just want to, talk a little bit about okay. So we covered a lot of ground. We talked about some of the challenges, what's led to the fatigue, the lack of trust, and and kind of how we move to operationalizing things, getting unstuck, and moving to more direct skill validation. But still, where do you even start? And so this I just wanna first ask you, Madeline Madeline, what would be your advice, your kind of simple practical advice for what organizations do to move from this this state of fatigue into, actual action for folks to get started? Yeah. I think, there's two things I would say. One is I like that model that you have. I think everyone should copy that model of the inferred and then the self reported and then the validated skills. I mean, I think that breaks it down very clearly, in a way that's not overwhelming for companies. I think the other piece on the stuck with skills is you don't need to go into your organization and do what we did fifteen years ago with competency models. You don't need to spend three to five years building out a skill strategy or what we used to do with competency models that, you know, you're looking at what you have, you're assessing that, you're building out a strategy, and then you're looking at what you need to invest in. I think that's the wrong approach. It's a very static approach to a skill strategy. Skill strategy should be very dynamic. It's changing all the time. If you go through that process for three to five years, you'll be even more frustrated because everything will have changed in your organization by the time you actually publish that new skill strategy and do that work manually. I think in in investing in technology, investing in AI is what moves the needle forward in a skill strategy. And, you know, you laid out the options that companies have, but, you know, that that's what does it. That's what get come that's what gets companies to actually move forward. The sitting in, you know, your office with your organization and figuring out how to move forward with the skills taxonomy or ontology or inventory that you wanna go forward with, is just too time consuming, and it will fatigue anyone. It fatigues me just talking about it. Yeah. And, so appreciate that and, promise we didn't compare notes to this answer beforehand, but, I had some similar thoughts on this. Just from what I've seen, and from where I've seen the fatigue setting in and from talking to customers and going to conferences, to Madeline's point, it has been like, we are now two years into our skills journey. And but it's and and, like, a lot of work's gone into it, but there's still nothing to show for it. It's still trying to get your arms around the entire organization and defining that taxonomy for the entire organization and the strategy. And so, really, that's, like, my advice is, like, I think too many have tried to boil the ocean, and it's just taking too long. And the organization is losing patient it's leading to fatigue, but the organization, is losing patience as well. So the advice would be is, like, start small and get wins. It doesn't have to be the perfect overarching strategy. And I think, Madeline, you you made such a great point because once you get the overarching strategy just perfect, well, you just you know, 50% of your jobs just changed and you're and you're kind of back to rethinking things. So it's like operate, be nimble, operate with speed, take some more, take some jobs or a job. Start small with a job that we know we can get it right here, or we have a pretty good idea what we're doing here. And typically, that's some lower level jobs, high volume. Start there. Learn from it. You're not gonna do it perfectly, but learn from it. And then the other key is to, document success. Like, like, coming back to the organization to, like, why should we adopt a skill strategy is coming back with, like, hey. Here's how we actually reduce turnover or increased, increased performance or enhanced diversity through the skills based strategy. Yeah. So, let's we got about nine minutes left. A few questions have come in. So let me, I will throw these out there and, Madeline, certainly, we can tag you. So related to, what's, I you know, a point that I just shared is, like, how do I get leadership buy in for assessments or doing skill validation? So I'll, I'll let you start on that one, Madeline. Yeah. I mean, I think the great thing about the assessment market, why I have always loved it, is because it's the outcomes based approach. Like, that's what assessments is able to provide. That an investment in in assessment is an investment in being able to show outcomes, and not a lot of other areas of, you know, HR tech, are able to say that. So I think that's certainly, a great place to start. I mean, I think every assessment provider, that is a credible provider can help and support that outcomes based approach even while you're building a business case for that. So I think that's one certainly one piece of the puzzle, but it's science. There's so little in talent acquisition that is science. Right? We have art and science. That's what talent acquisition is. There's a lot of art. There's a lot of guesswork. There's a lot of creativity, that's why it's such an amazing industry, it's why I love it. But this is the science piece of it and it needs to go hand in hand with the art and that's, you know, to me, that's a very powerful argument as well. But, you know, certainly lean into providers to help support the outcomes based approach. It's what your senior leaders wanna see. It's what your CFO wants to see. And, you know, again, there's not a lot of different areas of HR tech that can do that. Yeah. And I'll just, great points. And I'll just I'll be short and sweet with this because I kinda already touched on it. To get leadership buy in, one is, like, we need to move with speed. It can't be this two year kind of program of building the the framework. We need to move quick or we're gonna lose their interest and they're gonna lose patience with it, and then we need to show results. We need to say, like, hey. This is moving the needle and this is how we get so, that's always been the way that we get support for assessments is day one after putting something into place, the hiring manager and leadership is gonna say, is it working? And having the answer to this is how it's working for us is the best way, to get to get buy in. So a few more questions. We'll try to get to as many as we can. What can companies do to prepare for job skills changing in the future? So I love this question because they will absolutely change. It's such a great question because it's an important one. These jobs will absolutely change in the future. And I think that that's why, you know, this kind of dynamic approach to skills and thinking about validated skills allows for that. It's constantly evolving. It's looking at how, you know, your organization's changing, how skills are changing, and it's a work in progress as opposed to what a still static skill strategy is, which is just defining them, taking several years to do that, and letting it sit until you revisit it three years later. So, you know, it's it's an important question, but I think AI is dynamic. That's the power of AI, and that's the power of what, you know, I think we've been talking about with this validated skills approach. And and I would add and it is a great question that I would add to that. So, again, organizations need to be more nimble, more nimble in the way they think about their jobs, being able to think of them in terms of skills. Like, we've talked about all of that. But what we haven't talked about, because it really hasn't been the focus of this session, but, like, understanding your internal talent from a skills based perspective is, like, how do I prepare? What skills do I already have within my organization that then maybe they're not sitting in that function or it's adjacent, like, career paths now look less linear and they're more sideways. So opening that up and getting smart about the way that you understand and move your internal talent is a great way, to get started as well. That's a good question. I do want to read this one because I know I personally can go, a little bit overboard on bashing resumes. So, and and Taylor here is, I think, calling me or Asana a little bit and right to do so. So do you think that resumes will eventually not be required to apply for a role? It's difficult to imagine this not being involved in a hiring process. So I can start I made you start the last two ones, so I can speak to that. I think you're absolutely right. I don't see it necessarily going away. Again, when you think about back to the chart of inference, self report, and, and validated skills, there is still value in inferred and self report skills. And that's what you're getting from the resume. There are also jobs that, resumes are more important. Like, having a resume for a very entry level job or experience doesn't matter. I can see it going away there. But jobs that you need experience, people need to see you've done this before, resumes still certainly have a place there. So I would say less reliance overall for some jobs maybe go away, but I don't see it, at least in the near term, completely going away. And, in some applications, there is value. But, again, you're still not going to base your hiring decision on the resume. Okay. Let's see. We've got time for one more, from Jackie. With rising costs of going to school and increasing, what courses from Coursera and those free online sites, where you can enhance and add skills, are those important to employers? It's a good question. I Yeah. Go ahead. I could I mean, it is it's a fair question, and I would say, today, maybe not as important as it should be. I think as the world continues, to move into skills based hiring and an understanding of, like, this can't necessarily be, do you have this past experience and I don't rely on the resume. And I'm just looking for, do you have this skill to the extent that you can develop a skill? And sometimes you can with the course, but other times it's like the course's foundation and then I actually have to go do it. But I can go do it in my current role. There's different ways of applying and building that muscle that you started. So I do see that becoming more important and that we as we start to operationalize and actually do skills based hiring better and pull out those skills, that employers will start to look at that. Does this person have this skill, and have they, you know, like, seeing that you've taken a course and you've applied it, that that can be will become part of the equation. Yeah. I know. I think it's a it's a great question. It's a hard one to answer too because it depends on the role. I think, it's a very interesting time for candidates and job seekers right now because even with the skills based approach, they're trying to think and navigate through what they have. I mean, we're seeing a lot on candidate fraud. We're seeing a lot on a lot of different discussions. I think, you know, the the hard part too is as just as companies are evolving, the roles that a lot of candidates are being hired into will be evolving too. So, you know, I think it's it's not gonna hurt, but I don't know if it's always necessary as well. Alright. I think we're right on time, and that is the end of the questions that came in as well. So perfect. So a reminder to the audience, that, some of the data that we shared today from Madeline's and Aptitude Research's, report on skills, can be downloaded, in the q and a. It will also be included in the follow-up email that comes out. So wanna thank the audience, for attending, for your engagement, and, also thank you, Madeline, for joining me today to have this discussion. Thank you, Mike. And thank you everyone for attending. Thanks Thanks all.