Terry Bickham is a national learning director from Deloitte and Touche. Allison Rossett has written many books on training, including "Beyond the Podium, "First Things Fast and most recently "Job Aids and Performance Support - Moving from Knowledge in the Classroom to Knowledge Everywhere.
Access the archive of this webcast here
DC: Today we are here to answer your questions. We got a question by E-mail before the event today, so I thought I would start it off with that one, "How do I handle training needs assessment for soft skill training?
AR: This is something I've worked on many times in my life and also have written about in
"First Things Fast: A Handbook for Performance Analysis and in "Training Needs Assessment. The approach to training needs analysis would be no different whether it's technical skills or soft skills. Yes, your sources of information would be different, perhaps some of the questions might be a little bit different, the subject matter experts would be different, the model performers to whom you would look at would be different. But in terms of what you do in an assessment, it's really not any different.
There are three kinds of questions you want to ask. You want a picture of what greatness looks like from multiple perspectives including the customer who asked for the program, water-walkers (really successful performers), known best practices and published literature. For example, in leadership or communications or customer service there is already a lot of information that gives you a vivid picture of what greatness would look like.
Next, you want to ask, 'Where are they going right and where are they going wrong?' What is current performance? Have you been getting complaints from customers about customer service? Are there any problems that you see in the exit interviews comments from people about communication skills of supervisors or managers? You need the details about what is currently going on.
Third, especially in the area of soft skills, you need a picture about what will drive performance. What is currently impeding it and getting in the way? When we in training are involved the assumption is that good performance isn't happening because employees don't know how. But there are other reasons, like the organization doesn't honor the behaviors or the software doesn't work or you have competing priorities or there is no incentive and so on.
TB: Allison, I echo everything you said absolutely. I would go about it the same way as I would if I was having to put together technical training. Check to see what your corporate competencies might say about expectations for soft skills. You may have something, even if it's in a vision or mission statement, that says here is the kind of organization that we will becheck there and see what that portends for soft skills.
I have had better success in talking with management and line folks when instead of talking about 'soft skills' I talk about 'business skills' or 'interpersonal skills'; particularly the phrase 'business skills'; line folks understand what that means.
In getting information about what employees are currently doing, try to get some sort of access, (even if it's at a high rolled up level) to what sort of trends are in the annual performance reviews. What are supervisors saying about folks they supervise as far as skills that are needed? Then in turn find out what sort of feedback employees provide about their leaders. From that you can get a trend. At an organization I was in up until about a year ago, we got quite a bit of feedback from the unions on the types of soft skills our supervisors needed, what was lacking. That was a good source of data.
DC: Here is another audience question: What is in the latest technology or practices to keep participants engaged in E-learning?
TB: Something that I have seen recently in E-learning that I think has been very effective, has been getting away from a linear design. Not just going from page to page to page and maybe having some questions or roll overs or little pop ups. In trying to build our new E-learning courses we aim for what I call 'stay on the page' activities. This is where there is a chunk of information on a single page of E-learning in a web-based course. There is lots of activity just staying on that one page. It's more like a website with links and other activities that make it more of a scenario-based screen. There are fewer numbers of screens, but lots of rich activity right on one screen, so keeping people on one page longer rather than just paging through them all. That is something that I have seen that I really like.
AR: Persistence and engagement in E-learning is really a great topic. All of the data suggests we are moving to a reduced number of instructor-led courses. Instead there is more E-learning; and I like a big tent definition of E-learning. If you are interested in this you can just Google my last name Rossett and 'big tent' and it'll take you to some free articles on this subject of a broader definition of E-learningincluding knowledge bases and performance support tools and coaching.
The question is engagement. I think you have to ask the question about particular programs and why people are not engaged and persistent because the reasons vary. Sometimes it's a bad program, boring stuff. Sometimes it's because they had so much they can't find their way. In our new book we have some examples. IBM did a whole chapter devoted to how IBM handles having so many rich resources under the big tent of E-learning that they really had to put a lot of energy into the guidance systems, in helping people find what they needed because it wasn't a matter of scarcity, it was a matter of plenty and thus having good systems to help people get the stuff they want. So you want to ask why. Maybe it's a problem of unmotivated employees. Maybe it's a big old hassle to do the downloads that are involved. Maybe they don't have the proper software on the system.
The number one thing to me in persisting in E-learning is to give them (and ourselves) a really clear picture of what we are talking about. Are we talking about a 10-minute experience on line? Maybe a podcast which shows them a little bit about the new product and then as Terry was suggesting, hope they will go, jump off and look at some more detail. Or is it an eight-week supervisory course or project management class? These are very different and we have to use different strategies to get there.
But the number one thing to me about online learning is a match between what your employees need to do the work well and that the materials meet those needs. The other thing if you really want to get engagement going, is supervisory involvement in the story. They can't be considered an adjunct. They can't just send them to the class. They've got to be engaged in perhaps luncheon conversations surrounding the materials. I think that's it.
DC: If you are trying to teach a manager, not just a training specialist, how to be smarter about training, what are some of the rules of thumb you might give them?
AR: Anybody who is involved in training has banged their head against the wall because they develop people in face-to-face events but in the workplace there are supervisors who either ignore it, pooh pooh it, or do not advance it. The main thing that I would want to do is have the importance of training to be part of on-boarding for supervisors and managers, and have in their incentive program and in their job description that they are players on the learning team.
Managers must be engaged in continuous conversations with their people about their needs, and ensure that they are familiar with the resources to which they can turn. It's not just a matter where you know you need to be better at math and then have no clue as to what is inside the company or the agency or the organization to help you.
And then continue the conversation. The most effective learning occurs over time in context. Supervisors are the key, as we talk about blends and blended learning. Supervisors have got to be positive and engaged to make this happen.
TB: What we are doing is taking a very heavy classroom instructor-led based curriculum and repackaging that as much less in the classroom, by taking content and moving it into the workplace. Many things that we are going to be putting online are tools, job aids, performance support systems, supporting kinds of content that people can refer back to, not just something that they sit in the classroom and get exposed to and then may or may not be able to find or use it again on the job. So we are repackaging our approach to learning, not just in the classroom but into the workplace.
As Allison said, it's absolutely critical that the managers are involved in this process. Everything from being involved up front and helping us create the right content and validate the job aids and how the work gets done, but then enforcing it, the use of it on the job. I don't mean enforcing it in a prescriptive way but encouraging people to use the references, to use the tools, so when a junior consultant comes to a manager and says, "I have forgotten how to complete this process, what we would like the supervisor to do is say, "Here is where you can find the job aid that will walk you through it, try doing that and if you have any questions, I will be glad to help you, instead of saying, "Didn't you go to the class? You should know it.
The second part of the answer I would answer in a little different way. If a manager thinks that training is always the solution, we need to get the manager to re-think that approach. If someone's performance isn't meeting expectations then the manager may think obviously we need to send this person to training, but there is much more to it than that. So I would also encourage managers to not take a knee jerk approach to doing training first but instead ask, will training really help in this case or is there something else that we can do to get performance where we need it to be?
DC: You've moved towards an answer for one of the other questions we had and that is, what if you as a training professional know that training isn't going to work. If someone says, "Give us training on customer service? you may be thinking, "I can, but I don't think that's what you need, but it's a hard thing to say given that it's your job to give training.
AR: Is it your job to give training? I am not sure that it is. It's our job to help the organization and individuals in the organization to perform more effectively. If somebody asked for a customer service class I would want to know that in fact there are some real serious skills and knowledge issues that are getting in the way of customer service in which case training would be appropriate.
I need to do at least some quick analysis, some of those 'first things fast.' If my instinct suggests it's not a training issue then I would want to start asking more questions, and I would have to lead my customer through questions that I am asking him or her, and through data that I have gathered from talking to employees, to alert them to other kind of causes of poor performance like conflicting priorities, lack of incentive, or lack of clarity that was expected.
TB: The question is if someone comes to me and says, "I want a five-day training program, we budgeted this amount of dollars for it, and by the way we want it held in Orlando, then how do we handle those kinds of things? Well, we know that our job isn't necessarily just to put on the training programs, our job is to make a difference in the organization through the performance of our people. So, how do we respond to that request? The one thing that I don't do in practice is say, "I am not going to do that, because then all of a sudden I am an obstacle that folks need to go around and of course, you don't do that. What I do instead is tell them I will be glad to help. "Let's talk a little bit more about what it is we are trying to accomplish before we talk about a particular training program or how long it should be or how it much should cost; it's a little premature. Let's talk about what it is that you want to accomplish.
Sometimes the response to your comments will be, "I told you, a five-day training program in Orlando. Then I say, "Let's set that aside for just a second. I can certainly do that but I can't guarantee that it's going to meet your needs and then we may be having this conversation again in a few months or years. So, I am very straightforward with them, without turning them off. Sometimes we have to be defensive but I advocate an offensive approach, but by doing that, by turning what our clients want into something that they need.
DC: I find that very helpful. You are asking us to think first about defining what our role is in our own hearts and operationalizing that in a way that comes across as positive and professional. You are coming to the table equipped with methodologies and analytical tools and the body of knowledge that is actually going to be able to help the person.
TB: And I would say it wouldn't be professional on our part to just give them what they want. I know that happens a lot and sometimes it's done for survival or out of politics or for whatever reason, but I would say, we are not really being true to our profession if that's what we do.
AR: And also think about it like this, if you just give them what they ask for, and that's not really what was required to help them be better at customer service in the short run it pleases, but in the long run it certainly does not please. They are aware customer service hasn't improved. We are trading long-term contribution for a short-term nod and I think we can do better in this. We have been professionals for many years now, and have really tried to define ourselves not just by the number of the events we offer or butts in seats or hits on websites, but also by the difference we are making, whether it's customer satisfaction, repeat business, or if we work in a community college perhaps in reduction in dropouts, attrition, and so on.
DC: Can you discuss some best practices around evaluating training?
AR: We are working with a company right now on that very issue. I am working on a strategy document for a global evaluation measurement strategy. The problem with measurement is we don't do enough of it. We don't ask really important questions and we don't use enough sources. Historically, we go to the people who are in classes, which is one way we are developing people, and we say, "Did you like it, will it be useful, do you appreciate it? That's done in over 90% of training eventsthis is ASTD data. And it's been supported by an E-learning Guild study and there is lots of data on this.
The next typical question you ask in the Kirkpatrick model, and would make sense in any model, is 'Did they learn anything?' Now how often do you think that measure happens? The answer is just over 34% of the time, in just over a third of circumstances do organizations actually measure skills and knowledge. It stuns me how rare that is because working at a university we do try, not perfectly but we surely do try, to measure learning.
Then we think about when people go back to work, and we ask 'what do they do there?' We ask that 18% of the time. (Now remember what I am describing here was a model of training events and then transfer back to work. As we move to the model where there is more learning in the workplace, reference in the workplace, and coaching in the workplace, that particular question doesn't really make so much sense).
The last question is, 'Does it matter? Does it affect the business results?' Three or four percent of the time we get data on that. So basically, we don't have enough sources of information, we don't ask enough questions.
Shouldn't we be bringing evaluation and analysis together? For example, you offer a class in ethicsan instructor-led piece, and E-coaches on ethics, and online conversation, and knowledge-based policies and practices. You don't just want to measure the class, you want to measure participation in the online community, satisfaction with the E-coach and because you need to improve this system you also want to make sure there are more ethical decisions.
You want to take the longer view about results as you go forward. Are you doing evaluation there or are you doing analysis to make a better program and help your E-coaches and your instructors be more effective? The answer of course is analysis and evaluation are coming together. You can access a free article on this subject, I wrote about it in Training and Development in February 2007. Other people you should read, if you are interested in this topic, are some work by Robert Brinkerhoff, and the usual, Jim Kirkpatrick, Les Donaldson, Jack Phillips.
DC: Now over to you Terry, getting back to that measurement question. What are you actually doing?
TB: It is a great question. We are taking a measured approachno pun intended. We are first of all trying to make sure that we have got a great learning evaluation or learning measurement system in place for lower levels. Allison mentioned the Kirkpatrick levels and level one which was the reactions. We are a pretty large organization, so what we were trying to do is to make sure that we have got a foundation, first of all in getting reactions. Because if there is good data there, it isn't necessarily data that's going to drive our business decisions but it will certainly help us improve the learning that we are offering in many ways. So, we were working there and then we take a stair step approach.
For example, in all of our learning programs we want to gather that level one or reaction data. Moving up this ladder there are programs where we do want to get assessment data, we do want to find out if they have learned something at these programs. And then going up to the next step, if you picture a fewer number of programs, we are going to find out if they are actually using what it is that we have spent time and money on trying to build their skills. As we move up this evaluation ladder a last step there are a few other programs where we will extend considerable effort to see the impact. Because it is not easy to get the data at these higher levels of evaluation, you really need to focus on how you are going to get it, and in a meaningful way, not just gather data that isn't going to tell you what you need for a key number of programs.
For example, every year we run a new manager seminar. It's a week-long program for our folks that have just been promoted to manager. We have around 700 people attending this program. We want it to be a great program. We want to find out if we have been effective at helping them be better managers after they leave our program. But we may not expend that level of effort for something that's not as much of a milestone program.
DC: That makes a lot of sense because doing a very detailed evaluation for all your programs is just not something that you have the budget for.
TB: You might not get an ROI for it even if you had the budget. You might not get information back that's going to be useful and helpful for you.
DC: Do you have any favorite training stats to share?
AR: I look at numbers all the time, and I was just looking at some on the shift to more workplace based and independent learning. I like the ASTD State of the Industry Report which they put out every year. I like Training Magazine's October census study of what's going on in the field. Chief Learning Officer (clomedia.com) just put one out maybe a couple of weeks ago. E-learning Guild is doing a lot of work in the technology space on rolling up answers from people who come to their website and capturing data from them on what their practices are, what their needs are, what their favorite software systems are and so on.
TB: Training alone fails to change performance about 85% to 90% of the time. We need to be aware of that when we are imposing training where it may not be appropriate or training by itself where it may not be appropriate. And that one fact never ceases to amaze folks. So that's one of my favorites and another one right now that is very much a part of my world is that within the next couple of years, 50% of our workforce are going to be comprised of Gen Y folks. That means they will be under 30 by 2010. That has important messaging for how we do training, especially now with workers staying in the workforce longer. That means that we are going to have tri-generational learning issues that we are going to need to address in all of our programs. So we need to be aware of the generational differences and when 50% of your folks are going to be perhaps significantly different learners than the other half. We need to know how to address that.
DC: What catches your attention as a promising change in the way we in the training profession do business?
TB: I think we have been talking a lot about that. All the things we have been talking about measurement, about trends in e-learning, trends in learning in general.
I can tell you from my end that we have a very difficult time hiring learning professionals. The market seems to be very good for people who are in this profession and who are good at it. To me that's one of the indicators that portends well. It means that organizations are placing value in learning, that they are investing in learning, and that I hope it's a trend that will continue.
AR: It's a good marketplace for emphasis on the word professional. People who make good decisions, make decisions based on data, and don't apply the same strategy or technology to everything. They know everything isn't an opportunity for performance support tools or instructor-led training but rather they make measured decisions. They form partnerships in their organizations with the business units and they are client facing, they are focused on the result that the organization is trying to put in place. They join the team and advance the goals of the team, at the same time being fascinated with all that's involved in the learning and support.
That's nothing new, the main new thing, the main shift, is from the default being instructor led training. Of course it still is the dominant mode; about 65%, 70% of all training is still happening in instructor-led format. But it is steadily eroding and being replaced. The point is not that it's technology, although technologies are certainly advancing it, enabling it. The point is that we are providing education, information, and community and ideally supervisory support, closer to where the work gets done. We are moving from a world where you went to training and then we prayed that it would transfer, to a situation where resources are provided as they are needed, in which case transfer shouldn't be such a big deal at all.
DC: That's quite a profound change and I guess it ties into Terry's comment about the fact that unaided training fails so frequently. Let me ask another common question. The company is looking to save money and you feel sure they are going to hack money out of the training budget without any thought. What can you do?
TB: I would say if you haven't been concerned about addressing that all along, it may be too late. It gets back to what we are talking about - demonstrating the positive impact that training has on the organization. If the decision has already been made to cut the training budget, it may be a little too late to run back and start saying, "Wait a minute, look at all the things we have done, we have run these many classes or we have trained this many people. That's not the right information necessarily, and it may be too late. So what I would suggest is be proactive and show the impact of what it is that you do before the decision is made to cut the budget.
DC: Times are pretty good right now for most organizations so it's a good time to start gathering evidence of the value of training and communicating the value that I am adding.
AR: This takes us back to the question of measurement. What's the purpose of your unit? There is an array of possible purposes. Putting in skills and knowledge is one purpose but alignment between the individual and the organization is another purpose. There are more than a dozen possible purposes. Are you measuring any of those things, other than just the number of bodies and satisfaction with the experience? If you are only measuring number of bodies and satisfaction with the experience, it's entirely possible that a concerned bottom line oriented executive is going to think of other ways of getting butts in seats than having an internal learning organization. An internal learning organization means we are working together towards common results, not just learning results, but common organizational results.
DC: Is there a training story you would like to share?
AR: This happened to me many years ago. A learning leader, a proud fellow and nice guy, showed me a room that was filled with file cabinets. He pointed to the file cabinets, there were probably 20 of them, and he said, "Guess what is in there? I said, I don't know. He said, "All the evaluations for our classes. Remember, this was years ago, so it was 100% instructor-led. I was impressed with so many file cabinets, so many evaluations, asking employees if they like the class or not. I said, "What do you do with them? He said, "Most of the time we send them to our instructors so they can be enlightened, they read them, we hope.
There were no conversations with instructors. No rolling up of the data to pinpoint issues. No feedback that would lead to changing the curricula. What about looking for issues and following up afterwards with employees to see if they are able to use the skills or if there is alignment in the organization? Nothing. Basically, they were measuring bulk.
I ran into the same thing in a government organization a few years later where they too had a very fine roomnow of course, this is all online. But the question is, 'Are you asking good questions and then putting legs on the data, getting it out there so it helps people, instructors included, make better decisions?
DC: Terry, do you have any story that you would share?
TB: I have a short story under the banner of "Know Your Audience. In the early 90's, I was doing some training for the federal government and had been asked to go to Eastern Europe to work with some of the police and customs officials in a former Soviet country. They had just recently gotten their independence from the Soviet Union and were putting their own identity in place. I was really excited to do that and flew into this airport, where I was met there by the translator that the US embassy had provided for me. We filed in a car and drove four hours out into the middle of nowhere to this other town for the training to begin.
It was a week-long class. We got there Monday morning, bright and early, ready to start the class and of course no one in the class spoke English so the translator was translating for me. We made it through the morning and I noticed that the students and participants seemed a little stern. I thought maybe it's the culture. We broke for lunch and some time while I was eating my translator disappeared and I was trying to find him. He was nowhere to be found then this young college guy showed up in the classroom and spoke English, thank goodness, and said, "I am going to be your translator for the rest of the week. I said, "Why is that, what happened to the other guy? He said, "He was translating everything that you are saying into Russian. Of course, all these people speak Russian, but they do not want to hear it. They want to hear you in their native language. It could have been a horrible thing. But I was able to recover and get back in their good graces for the rest of the week and ended up being a success.
