Theory v. Reality: There’s Room for Both

learning mapThis is a post from a Boise State student (I am an alum) in response to a webinar I participated in during 2010.  I welcome the conversation and have inserted my comments in blue.

“I feel like I should preface my write-up below by explaining that while I was listening to the webinar and reviewing the transcripts, I really was looking for an aha moment when I could apply what I had learned in class with what was being done in the field. While I had “my aha moment”, in that I was able to make this connection, it was not in the manner I was expecting. I hope I don’t seem like I am being too critical, as there was a lot of good information presented in the webinar. I am choosing to share this as my “aha” moment, because I think it makes a good correlation between the material learned in class and how it may be applied in the field. Also, not sure if I needed to add references, but since I can use the practice, I made an attempt. 

Elliot Rosenberg touched on several topics during his webinar with Dr. Steve Villachica on October 11, 2010. These topics included discussions on cross generational trainings and the fact that in 2020 there will be five generations in the workplace. However, the part that captured my attention was his discussion on training evaluations. Typically I find it nice to see how class material applies to the everyday world, but I was really confused and disappointed in what Rosenberg had to say about training evaluations. Here is the excerpt I am referring to in which Rosenberg describes the use of training evaluations (as copied from the webinar transcript): 

If Level Ones is a measure, I’m right there. Even Level Twos—but honestly, I don’t really care about level ones and twos—you have to understand it to see that if your project is at least on target, what really matters is the level threes and fours. And let’s not put all the stock into Kirkpatrick’s Levels of Evaluation; it’s possible lots of variable could impact it. The bottom line is, for me, are we making more sales? Are we reducing our employee turnover, and are we increasing customer satisfaction? Those are the three measures that guide the success of our efforts. It has nothing to do with levels, quite frankly—it’s all about the business. 

I feel like on one hand he is explaining how he uses training evaluations in his organization which follow Kirkpatrick’s theory, but then he makes statements like, “I don’t really care about level ones and twos..” “Let’s not put all the stock into Kirkpatrick’s levels of evaluation…” He identifies three questions he wants to answer at the end of a training session, but then follows up with the statement that they have nothing to do with levels. This is where I disagree. Rosenberg describes his bottom line as answering the following 3 questions: 

  1. Are we making more sales? 
  2. Are we reducing employee turnover? 
  3. Are we increasing customer satisfaction? 

I would argue that by answering the three questions above he is in fact completing a Level 4 results evaluation. Chyung (2008) describes a Level 4 evaluation as a look at results on the organization and examines if the training resulted in a behavior change that positively affected the organization. If the training allowed the sales managers to make more sales, reduced employee turnover and increased customer satisfaction, I would think that the training positively affected the organization. On the other hand if you had “No” answers to any of the questions – you might think there was a negative impact on the organization. Either way, the results from these three questions still represent a Level 4 Results evaluation since they identified how the organization was impacted by the training.”

Kirkpatrick’s Levels of Evaluation provide the framework from which ISD professionals can measure their work and, hopefully, impact.  However, without context for those measurements, they will mean nothing for your client.  

Now, in my case, my department is not funded by HR but, rather, by the sales organization.  While HR might find it OK to say that participants liked the training (L1) and they were able to pass a knowledge test after the class (L2), for a sales organization there is no link to their objective which is to make more sales.  And, if the learning organization can’t impact those results, why have them around?

If the implication is that I actually achieve a L4 result if I can impact all three questions, that might not work with someone who is skilled in ISD.  A nearly impossible task of isolating all of the variables that could impact L4 results is critical.  However, if I could imply that training had something to do with increased sales, reduced turnover, and improved customer satisfaction then we could have that discussion.  If you believe the 70/20/10 rule for formal vs informal learning (only 10% of learning is accomplished via a formal setting (ILT, for example), then it is only reasonable to assume that the other 90% is accomplished outside of the formal setting (20% from coaching and observations and the other 70% from simply gaining experience by doing the job), then you can see the diminished role that Kirkpatrick’s concepts have on learning, over the long term.  Even if you applied this line of thinking to the academic world, you should draw the same conclusions (IMHO).  

You have to start with Kirkpatrick but you must not end there.  Thoughts?

Why I Became a Certified Performance Technologist (CPT)

ISPI Atlanta logoWhy did you decide to become a Certified Performance Technologist (CPT)?

When the CPT designation was introduced, it offered practitioners an opportunity to meet new standards of professional performance and expectations.  It also offered me instant credibility in the industry and with those with whom I work.  I guess you could equate the CPT designation with similar designation from the accounting profession (CPA), financial planning (CFP), along with a host of others.

What does the certification offer you?  

Certification offers peer recognition along with a commitment to continuously learn.  It implies to the industry that I’ve met specific standards and am “certified” in my level of skill and knowledge.  It does not get you more money but, via enhanced credibility, it most certainly can.

Have you seen any results because of your certification? What are they/or why not?

My peers understand the process and, therefore, it established a benchmark of knowledge.  I haven’t received any financial benefit from the certification but continue to maintain it as part of my professional development and industry insight.

Would you become certified again?

I’ve renewed my certification several times, each by meeting the established criteria.  Further renewals will be determined at the time that it is due.  There is a financial cost to renew, and it must be weighed with the benefits.

What do you see as the difference between the ISPI certification and the American Society for Training and Development (ASTD) certification?

To me, ISPI is more of a practical application.  It gets to the heart of the comparison between the two organizations.  ISPI is focused on performance improvement, while ASTD is more generic in its focus on training.

Still not sure if CPT is right for you? Read ISPI’s frequently asked questions.

Risks Worth Taking

I’ve been at this social thing for about two years and, during that time, I’ve picked up on a few things.  One of them is to take some risks.  Here are a few of them:

  1. I put myself out there.  Personally on Facebook, professionally on Twitter.  LinkedIn, well, not so much but it just follows Twitter.  So far, I haven’t been sued, slandered upon, or embarrassed.  Still, I’ve only been at this for two years.
  2. I’ve found some real interesting people who know what they’re talking about.  And, even if I don’t agree with them, it opens up a discourse in the name of a shared interest.  No back and forth emails between just the two of us but, rather, a global conversation.  I like it.
  3. There are some odd folks out there.  Those with tens of thousands of tweets, following lots of people but zero followers.  You’ve got to keep your guard up.
  4. I would classify myself as an “early adopter” but, with respect to social networking, I’ve got some catching up to do.  Technology is not the issue but, rather, consistently challenging the creative person within.
  5. Finally, I am pushing the envelope with those with whom I work.  Yeah, there are risks but failure to at least try would be disadvantageous to all.

What social networking risks have you taken? Respond in the comments section!

Internet Time Alliance: Working Smarter (March 2012)

Thanks to the Internet Time Alliance for this content.

The Internet Time Alliance helps its clients understand and embrace complexity and adopt new ways of working and learning.

“When I Grow Up — Work Version”

From the Kevin Jones series of videos — which does videos to explain complex ideas more simply.  This one, however, seems pretty straightforward.

When we were young we would never have thought these things. Now, too many of us live it. When we take a step back and do a sanity check, we realize that there has to be a better way. And, of course, there is — but too often we don’t feel there is a way to make improvements. But I believe we can.

This was inspired by the ad of 1999.

Here’s the original:

Training, social media books worth a read

Here are some good, industry-related books that have recently been published that I thought I would share.


  1. Informal Learning: Rediscovering the Natural Pathways That Inspire Innovation and Performance by Jay Cross
  2. A New Culture of Learning: Cultivating the Imagination for a World of Constant Change by John Seely Brown and Douglas Thomas
  3. The New Learning Architect by Clive Shepherd
  4. The 2020 Workplace: How Innovative Companies Attract, Develop, and Keep Tomorrow’s Employees Today by Karie Wilyard and Jeanne Meister
  5. Waiting for “Superman”: How We Can Save America’s Failing Public Schools by Carl Weber
  6. Working Smarter Fieldbook, September 2010 edition by Jay Cross

Social Media

  1. The New Social Learning: A Guide to Transforming Organizations Through Social Media by Marcia Conner and Tony Bingham
  2. Social Media for Trainers: Techniques for Enhancing and Extending Learning by Jane Bozarth
  3. Socialnomics: How Social Media Transforms the Way We Live and Do Business by Erik Qualman
  4. The Facebook Effect: The Inside Story of the Company That Is Connecting the World by David Kirkpatrick
  5. Social Learning Handbook by Jane Hart

50 Quotes About Learning

I love these!

  1. “The illiterate of the 21st century will not be those who cannot read and write, but those who cannot learn, unlearn, and relearn.” — Alvin Toffler
  2. “I have never in my life learned anything from any man who agreed with me.” — Dudley Field Malone
  3. “Always walk through life as if you have something new to learn and you will.” — Vernon Howard
  4. “Education consists mainly of what we have unlearned.” — Mark Twain
  5. “I am learning all the time.  The tombstone will be my diploma.” — Eartha Kitt
  6. “It is what we think we know already that often prevents us from learning.” — Claude Bernard
  7. “It’s what you learn after you know it all that counts.” — Harry S Truman
  8. “You can teach a student a lesson for a day; but if you can teach him to learn by creating curiosity, he will continue the learning process as long as he lives.” — Clay P. Bedford
  9. “Life is like playing a violin in public and learning the instrument as one goes along.” — Samuel Butler
  10. “We now accept the fact that learning is a lifelong process of keeping abreast of change. And the most pressing task is to teach people how to learn.” — Peter Drucker
  11. “The aim of education should be to teach us rather how to think, than what to think — rather to improve our minds, so as to enable us to think for ourselves, than to load the memory with the thoughts of other men.” — John Dewey
  12. “Wise men learn by other men’s mistakes, fools by their own.” — Unknown
  13. “By three methods we may learn wisdom: First, by reflection, which is noblest; Second, by imitation, which is easiest; and third by experience, which is the bitterest.” — Confucius
  14. “Life is a learning experience, only if you learn.” — Yogi Berra
  15. “Wisdom is learning what to overlook.” — William James
  16. “That is what learning is. You suddenly understand something you’ve understood all your life, but in a new way.” — Doris Lessing
  17. “Learning is not a spectator sport.” — D. Blocher
  18. “Anyone who stops learning is old, whether at twenty or eighty. Anyone who keeps learning stays young. The greatest thing in life is to keep your mind young.” — Henry Ford
  19. “We learn more by looking for the answer to a question and not finding it than we do from learning the answer itself.” — Lloyd Alexander
  20. “Smart people don’t learn… because they have too much invested in proving what they know and avoiding being seen as not knowing.” — Chris Agyris
  21. “I never teach my pupils; I only attempt to provide the conditions in which they can learn.” — Albert Einstein
  22. “All the world is a laboratory to the inquiring mind.”  — Martin H. Fischer
  23. “Nothing that is worth knowing can be taught.” — Oscar Wilde
  24. “If you hold a cat by the tail you learn things you cannot learn any other way.” — Mark Twain
  25. “I hear and I forget. I see and I remember. I do and I understand.” — Confucius
  26. “I am always doing that which I can not do, in order that I may learn how to do it.” — Pablo Picasso
  27. “We learn geology the morning after the earthquake.” — Ralph Waldo Emerson
  28. “Man’s mind stretched to a new idea never goes back to its original dimensions.”  — Oliver Wendell Holmes Jr.
  29. “Learning is not attained by chance. It must be sought for with ardor and attended to with diligence.” — Abigail Adams
  30. “No one as ever completed their apprenticeship.” — Johann Wolfgang Von Goethe
  31. “A man who reads too much and uses his own brain too little falls into lazy habits of thinking.” — Albert Einstein
  32. “All learning has an emotional base.” — Plato
  33. “Curiosity is the wick in the candle of learning.” — William A. Ward
  34. “I’ve known countless people who were reservoirs of learning, yet never had a thought.”  — Wilson Mizner
  35. “Learning isn’t a means to an end; it is an end in itself.” — Robert Heinlein
  36. “Learning is not compulsory … neither is survival.” — W. Edwards Deming
  37. “It is what we know already that often prevents us from learning.” —Claude Bernard
  38. “Everyone and everything around you is your teacher.” — Ken Keyes
  39. “You live and learn. At any rate, you live.” — Douglas Adams
  40. “Live as if your were to die tomorrow. Learn as if you were to live forever.” — Gandhi
  41. “Reading furnishes the mind only with materials of knowledge; it is thinking that makes what we read ours.” — John Locke
  42. “One of the reasons people stop learning is that they become less and less willing to risk failure.” — John W. Gardner
  43. You aren’t learning anything when you’re talking.” — Lyndon B. Johnson
  44. “Almost anything can become a learning experience if there is enough caring involved.” — Mary MacCracken
  45. “Never discourage anyone … who continually makes progress, no matter how slow.”  — Plato
  46. “Being ignorant is not so much a shame, as being unwilling to learn.” — Benjamin Franklin
  47. “Supposing is good, but finding out is better.” — Mark Twain
  48. “Develop a passion for learning. If you do, you will never cease to grow.” — Anthony J. D’Angelo
  49. “In doing we learn.” — George Herbert
  50. “It is possible to store the mind with a million facts and still be entirely uneducated.”~ Alec Bourne


Content Consumption: 11 apps I enjoy

I’m beginning to get more and more efficient with all of the information that is out there. It’s impossible to consume it all so you really have to just pick what you like and move on.

Sharing: Besides Facebook (personal stuff) and Twitter (professional stuff), I use to aggregate all of the Twitter posts of people that I follow. Those with similar interests as mine will find this daily publication as a means to simply have this content pushed to them. Click here and hit “Subscribe” if you are interested in reading my daily publication.

News: I use iPad for my main content consumption tool. There are three news aggregators that I use: Flipboard, Zite and Pulse. All have unique elements and none of them duplicate each other entirely. Flipboard and Pulse pull from Google Reader, Twitter, and Facebook. Zite only pulls from Google Reader and Twitter but also gets “smarter” as you use it over time. I also subscribe to AppAdvice for information about apps.

Music: iPod or Pandora

Note taking: These are plentiful and all dependent upon style. I’m currently using Awesome Note. I also have a keyboard dock that makes taking notes over a long period of time very, very efficient and effective. Reliance on the virtual keyboard for an extended period of time is not a good idea.

Translations: Goolge Translate is pretty cool.

Productivity: As with many apps, these are plentiful as well. From financial bill paying apps to photoshop apps, there are plenty of tools to make your online experience work.

Books: iBooks and Kindle. Actually, I like the Kindle app better for its sharing potential and a wider inventory of choices.

I have nearly 100 apps (just about all of them were FREE) but only use a handful of them. I would bet that is the same for most users. So, what are you using and how are you using it?

Disney World: Where Tomorrowland is now Historyland

Having spent the last few days there, I realized the great Disney organization can’t keep up. Some of the exhibits at Tomorrowland have been there for, well, a while. True, some have been updated but, for the most part it’s all kinda old news. Even the Carousel of Progress was updated but no iPad, no cell phones … virtual reality was the primary addition.

EPCOT is further along with space travel stuff. Cool (and fast!) rides from GM, plus some nice simulators are the big additions. But, the World Showcase? Many country exhibits are still showing movies that are clearly dated.

I can’t even imagine the effort it takes to pull all this off. It’s still impressive.

Just a word of advice to those with small children: they’re not learning anything when they can’t even walk. Wait until they are older and they, and you, will enjoy it a lot more.

Then, go back without them!