The Boggs e-Learning Chronicle

Book on Training Evaluation....

Found this up on Jim Kissane's blog...

How Do They Know They Know?: Evaluating Adult Learning

by Jane Kathryn Vella, Jim Burrow, Paula Berardinelli

Synopsis: "Applies principles and concepts of popular education to evaluation?a critical piece in program development and training with adults. Using real-life case studies, the book shows how the model works in a variety of settings to help trainers evaluate adult learning."

elearning	learning management systems
elearning podcasts	Competency Management

Dave Boggs
SyberWorks, Inc.
SyberWorks e-Learning Podcast Series
SyberWorks LMS e-Learning Implementation Podcast Series
e-Learning Lingo Podcast Series
SyberWorks Training Center Learning Management System

The Learner's Perception of e-Learning Quality

Quality of e-Learning: An Analysis Based on e-Learners’ Perception of e-Learning

by Rengasamy Elango, Vijaya Kumar Gudep, and M. Selvam Majan University College, Sultanate of Oman Skyline College, Sharjah, United Arab Emirates Alagappa University, Tamil Nadu, India

Abstract:

e-Learning, of late, has been witnessing an unprecedented expansion as an opportunity for higher education. This expanding alternative mode calls for ensuring and imparting a sound and qualitative education. The present study made an attempt to investigate the issues related to the quality dimensions of e-learning. Our results revealed the presence of both strengths and weaknesses in the e-learning system. It is interesting to note, that the e-learners have expressed diverse opinions with regard to administrative issues, instruction materials, instructors’ support, viper sessions (VIPER, Voice Internet Protocol Extended Reach is a software which helps interactive learning through the Internet) , grading and assessment. The findings of the study further demonstrate that if the concept of e-learning is imparted with a better approach and perspective, the reach will be phenomenal. This study reiterates the relevance of imparting qualitative education through e- learning.

elearning	learning management systems
elearning podcasts	Competency Management

Dave Boggs
SyberWorks, Inc.
SyberWorks e-Learning Podcast Series
SyberWorks LMS e-Learning Implementation Podcast Series
e-Learning Lingo Podcast Series
SyberWorks Training Center Learning Management System

Ethics Questions Which Underlie Most e-Learning Research

Ethical Issues in Qualitative E-Learning Research by Heather Kanuka and Terry Anderson

Abstract:

"In the mid 1980s education researchers began exploring the use of the Internet within teaching and learning practices, now commonly referred to as e-learning. At the same time, many e-learning researchers were discovering that the application of existing ethical guidelines for qualitative research was resulting in confusion and uncertainty among both researchers and ethics review board members. Two decades later we continue to be plagued by these same ethical issues. On reflection on our research practices and examination of the literature on ethical issues relating to qualitative Internet- and Web-based research, the authors conclude that there are three main areas of confusion and uncertainty among researchers in the field of e-learning: (a) participant consent, (b) public versus private ownership, and (c) confidentiality and anonymity."

elearning	learning management systems
elearning podcasts	Competency Management

Dave Boggs
SyberWorks, Inc.
SyberWorks e-Learning Podcast Series
SyberWorks LMS e-Learning Implementation Podcast Series
e-Learning Lingo Podcast Series
SyberWorks Training Center Learning Management System

The Impact of e-Learning Benchmarks

Benefits of e-Learning Benchmarks: Australian Case Studies
by  Sarojni Choy, Queensland University of Technology

Abstract:

In 2004 the Australian Flexible Learning Framework developed a suite of quantitative and qualitative indicators on the uptake, use and impact of e-learning in the Vocational Education and Training (VET) sector. These indicators were used to design items for a survey to gather quantitative data for benchmarking. A series of four surveys gathered data from VET providers, teachers, students and their employers. The data formed baseline indicators that were used to establish organisational goals and benchmarks for e-learning. These indicators were the first known set for benchmarking e-learning in Australia.

The case studies in this paper illustrate ways in which VET providers have approached e-learning benchmarking, the benefits achieved and the lessons that they learned. The cases exemplify how VET providers have adapted the baseline indicators, how the indicators informed organisational plans and e-learning outcomes. The benefits of benchmarking are categorised under three purposes: reporting, performance management, and service improvement. A set of practical strategies is derived from the cases for consideration by other organisations interested in benchmarking e-learning services.

elearning	learning management systems
elearning podcasts	Competency Management

Dave Boggs
SyberWorks, Inc.
SyberWorks e-Learning Podcast Series
SyberWorks LMS e-Learning Implementation Podcast Series
e-Learning Lingo Podcast Series
SyberWorks Training Center Learning Management System

Training Evaluation Primer....

While this article is from 2002, it offers some foundational information on  training evaluation and assessment.  The article looks at  six different approaches to training evaluation.

Approaches to Evaluation of Training: Theory & Practice  by Deniz Eseryel

Abstract:

"There is an on-going debate in the field of evaluation about which approach is best to facilitate the processes involved. This article reviews current approaches to evaluation of training both in theory and in practice. Particular attention is paid to the complexities associated with evaluation practice and whether these are addressed in the theory. Furthermore, possible means of expediting the performance of evaluations and expanding the range and precision of data collection using automated systems are discussed. Recommendations for further research are also discussed."

elearning	learning management systems
elearning podcasts	Competency Management

Dave Boggs
SyberWorks, Inc.
SyberWorks e-Learning Podcast Series
SyberWorks LMS e-Learning Implementation Podcast Series
e-Learning Lingo Podcast Series
SyberWorks Training Center Learning Management System

The Truth About Training Assessment...

On his Will at Work Learning Blog, Will Thalheimer has some interesting results on a straw poll he took on  assessment....

"In a webinar this month (December 2006), I asked a group of about 100 e-learning professionals, what was the highest level of assessment they did (based on Kirkpatrick's Four Levels) on their most recent learning-intervention development project.

• 11% said they did NO evaluation
• 26% said they did Level 1 smile sheets
• 48% said they measured Level 2 learning
• 15% said they measured Level 3 on-the-job performance
• 0% said they measured Level 4 business results (or ROI)."

This is not at all surprising to me.  Training resources are strained, and very few companies allocate an appropriate amount of manpower, time, and money in their budget to actually see if their programs work so they can be tied to  Kirpatrick's Level (4).

Assessment Mistakes by E-Learning Developers

elearning	learning management systems
elearning podcasts	Competency Management

Dave Boggs
SyberWorks, Inc.
SyberWorks e-Learning Podcast Series
SyberWorks LMS e-Learning Implementation Podcast Series
e-Learning Lingo Podcast Series
SyberWorks Training Center Learning Management System

Using Web Analytics To Evaluate Instructional Design

One method for evaluating e-learning content is using web analytics.  This is a short pdf (5 pages) on using web analytics  in  instructional design evaluation for  e-learning development.   This is not a new idea by any means and many organizations have used this along with other ways to evaluate e-learning content for a while now (since the  mid 90's, of course, e-learning really has even earlier roots, but that is not what this post is about) but the information in the pdf provides a good basic primer for those who are just starting out in e-learning development.

A Model 0f Online Instructional Design Analytics by Kenneth W. Fansler and Rodney P. Riegle, PHD

Abstract:

"The objectives of this paper are 1) to provide data-driven example instructional design evaluation for online course designers and administrators,  and  2) to provide a theoretical framework of instructional design analytics that result in a deeper understanding of online teaching and learning."

elearning	learning management systems
elearning podcasts	Competency Management

Dave Boggs
SyberWorks, Inc.
SyberWorks e-Learning Podcast Series
SyberWorks LMS e-Learning Implementation Podcast Series

PDF on Computer-Based e-Learning Assessment

From the Journal of Technology, Learning, and Assessment, Computer-Based Assessment in E-Learning: A Framework for Constructing "Intermediate Constraint" Questions and Tasks for Technology Platforms

Abstract:

"Technology today offers many new opportunities for innovation in educational assessment through rich new assessment tasks and potentially powerful scoring, reporting and real-time feedback mechanisms. One potential limitation for realizing the benefits of computer-based assessment in both instructional assessment and large scale testing comes in designing questions and tasks with which computers can effectively interface (i.e., for scoring and score reporting purposes) while still gathering meaningful measurement evidence. This paper introduces a taxonomy or categorization of 28 innovative item types that may be useful in computer-based assessment. Organized along the degree of constraint on the respondent’s options for answering or interacting with the assessment item or task, the proposed taxonomy describes a set of iconic item types termed “intermediate constraint” items. These item types have responses that fall somewhere between fully constrained responses (i.e., the conventional multiple-choice question), which can be far too limiting to tap much of the potential of new information technologies, and fully constructed responses (i.e. the traditional essay), which can be a challenge for computers to meaningfully analyze even with today’s sophisticated tools. The 28 example types discussed in this paper are based on 7 categories of ordering involving successively decreasing response constraints from fully selected to fully constructed. Each category of constraint includes four iconic examples. The intended purpose of the proposed taxonomy is to provide a practical resource for assessment developers as well as a useful framework for the discussion of innovative assessment formats and uses in computer-based settings."

Dave Boggs
SyberWorks

Intermediate Factors as Performance Drivers in e-Learning

Tony Karrer, on his e-Learning Technology blog has a post about Intermediate Factors In Learning that talks about how intermediate factors can become performance drivers.   He makes some good points.  I would say that depending upon the corporations collective culture, the management and learning style of the specific training department head will have a lot to do with their approach and then program management. 

Some managers, departments, or organizations won't even consider intermediate factors until they have some sort of benchmark, i.e. what the organization sees as success with the "product type" focus of how training impacts job performance.  I cannot say for sure, but my hunch is that more organizations, small company or large, are like this than the other way.  Sometimes folks just need a place to start.

I think Tony's trying to address an organization that may only  have a product focus which cares exclusively about the end result, i.e. how the training impacts job performance.  I think the big thing he is saying here is not to stop there.  He is right, some organizations do, and are perfectly happy with it. 

Dave Boggs
SyberWorks

Training Mistakes.......

Its been said that for every one negative, you should try to mention several positives with it to soften the blow.....but for those training professionals who are not faint of heart........Elliot Massie has posted the results of one of his surveys on training mistakes....... Mistakes.

Many of these so called 'mistakes' can happen in even the best run companies.  To avoid these mistakes, check out our e-Learning Best Practices white paper.  The whitepaper is chalk full of  proven e-learning / learning management systems implementation tactics that will help guide you through a successful e-learning program launch.

David Boggs
SyberWorks