On 2016 Jan 29, Martin Pusic commented:

Thank you for this insightful review - we're glad that the article created such a rich discussion. Here are a couple of other thoughts:

. "different tracks for different learners" - what a learning curve makes manifest is the time component of an assessment. As a medical educators, we have the privilege of teaching highly motivated learners who almost always get over whatever bar we set for them. If we grade ourselves as teachers by counting how many learners get over the bar, it is easy to perceive ourselves as successful; however, if instead we grade ourselves on the SLOPE of the learning curve, now we have a metric that challenges us to grade our efforts in terms of learning efficiency, which is amount of learning per unit of learning effort expended. This does three good things: 1) it orients us towards maximizing the most precious student commodity - time; 2) it prompts educators to consider more closely the PROCESS of learning as that's how you improve the slope and 3) it allows us to use the variability in paths/slopes to learn the best ways of teaching and learning. So it may be that we do not need customized learner development charts, as well as those work for pediatrics, but rather to learn from those outliers who fall away from the average curve so as to feed that back into the system to improve the learning for everyone.

.in the "life-cycle of clinical education" we would also encourage you consider the asymptote. The asymptote defines the "potential" of a learning system. "How good can we possibly be, if we used this system an infinite number of times?" Improving the slope means we get people up to competence more efficiently. Improving the asymptote means we get even better competence. In some cases we only need "x" amount of minimum competence and we're fine (think hand washing); but in most areas of medicine, we can always do better. The path to competence is all-important, but our learning systems would do well to also map out the path from competence to excellence, defined as being the very best any of us can be. The asymptote, along with the very shallow slope of the learning curve as it approaches it, gives us an idea of what excellence takes.

On 2016 Jan 28, Clinton K Pong commented:

Journal club review from a conference for Educators in Health Professions:

Pusic et al present a useful argument for the development and application of Learning Curves in Clinical Education. The learning curve graphs a mathematically modelled relationship between effort and competence. The authors suggest applications to practice citing work in radiological interpretation learning and the limitations of the work to date. At our journal club, we further considered learning curve application in the life-cycle of clinical education.

Theme 1: Undergraduate Education - the Steep Learning Curve of Med Ed

Learning curves may follow & identify the learner throughout their learning from undergraduate time onwards. Early engagement with validated learning curves, if available, across multiple domains, may have the potential to inform the learner and assist in his/her application to education early in their career. Learning curves may become a useful tool in assisting the learner-in-difficulty but challenges us to consider if there is long-term benefit of remediation in medical training.

Theme 2: Graduate Professional Training - Trainees falling off the "Growth Chart"

We considered how generalizable learning curves would be in the specialties where complex communication/perception is the competence required – eg the learning of elderly medicine rather than the more measurable accuracy/time-taken per task.

Does deliberate observation of practice confound performance in learning curves versus real-time learning in the clinically chaotic environment?

The evidence presented (Fig 7) suggested to us that a competency-based curriculum could result in different tracks for different learners – would it be ethical to continue with a time-based curriculum with the cost implications thereof if the majority of learners reach competency in a shorter than prescribed time or the corollary. Are there opportunity costs in overtraining the accelerated learner? How would programs and health systems respond/accommodate and support the learners who requires more/less than standard time to complete?

The curve may have a greater relevance in self-assessment metrics than formal assessment. Its creation informs reflective practice/analysis/germane learning and ideally would be used early in the training cycle. There may be risk in adding to the burden of extraneous cognitive load if performance anxiety mitigates the safety of the learning environment.

The authors acknowledge that there is variation in the starting point and slope of the learning curve that mostly cannot be controlled. In time, there may be evidence to describe if a less steep slope implies a perpetual shift in the curve to the right compared to peers or whether performance may suddenly shift the curve to the mode or even to the left as learning breakthroughs occur? The cohort is too small to usefully study these variations in this domain of learning.

Prediction models would be useful especially if wider work supports wide variations in trajectories of learning in complex clinical tasks.

What is happening in the area under the curve above the standard competence – Do these learners use and practice transferable skills that keep them ahead on the next learning curve?

Should we have customised learner development charts in multiple domains from the analogy of paediatric development milestones or growth curves – correction for some confounding factors could be achieved by customisation – for example - gender, age, ethnicity, population of reference – could these be used in clinical education to generate ‘growth of learner performance’ curves. Sub-analysis may lead to greater understanding of the non-thriving learners - the “under effort for time” learner versus the “under performance for effort” learner in the analogy of the skinny child versus the stunted child!

Theme 3: Faculty Development - "Cultivating Deliberate Expertise"

We considered how learning curve analysis could inform medical registration policy for re-certification if the decay curves were further developed. The unit time for performance degradation may not be identical in each cycle for revision learners. The theory of spacing learning activity to afford deeper learning over time may contribute to determining the appropriate cycle time.

On 2016 Jan 28, Clinton K Pong commented:

Journal club review from a conference for Educators in Health Professions:

Pusic et al present a useful argument for the development and application of Learning Curves in Clinical Education. The learning curve graphs a mathematically modelled relationship between effort and competence. The authors suggest applications to practice citing work in radiological interpretation learning and the limitations of the work to date. At our journal club, we further considered learning curve application in the life-cycle of clinical education.

Theme 1: Undergraduate Education - the Steep Learning Curve of Med Ed

Learning curves may follow & identify the learner throughout their learning from undergraduate time onwards. Early engagement with validated learning curves, if available, across multiple domains, may have the potential to inform the learner and assist in his/her application to education early in their career. Learning curves may become a useful tool in assisting the learner-in-difficulty but challenges us to consider if there is long-term benefit of remediation in medical training.

Theme 2: Graduate Professional Training - Trainees falling off the "Growth Chart"

We considered how generalizable learning curves would be in the specialties where complex communication/perception is the competence required – eg the learning of elderly medicine rather than the more measurable accuracy/time-taken per task.

Does deliberate observation of practice confound performance in learning curves versus real-time learning in the clinically chaotic environment?

The evidence presented (Fig 7) suggested to us that a competency-based curriculum could result in different tracks for different learners – would it be ethical to continue with a time-based curriculum with the cost implications thereof if the majority of learners reach competency in a shorter than prescribed time or the corollary. Are there opportunity costs in overtraining the accelerated learner? How would programs and health systems respond/accommodate and support the learners who requires more/less than standard time to complete?

The curve may have a greater relevance in self-assessment metrics than formal assessment. Its creation informs reflective practice/analysis/germane learning and ideally would be used early in the training cycle. There may be risk in adding to the burden of extraneous cognitive load if performance anxiety mitigates the safety of the learning environment.

The authors acknowledge that there is variation in the starting point and slope of the learning curve that mostly cannot be controlled. In time, there may be evidence to describe if a less steep slope implies a perpetual shift in the curve to the right compared to peers or whether performance may suddenly shift the curve to the mode or even to the left as learning breakthroughs occur? The cohort is too small to usefully study these variations in this domain of learning.

Prediction models would be useful especially if wider work supports wide variations in trajectories of learning in complex clinical tasks.

What is happening in the area under the curve above the standard competence – Do these learners use and practice transferable skills that keep them ahead on the next learning curve?

Should we have customised learner development charts in multiple domains from the analogy of paediatric development milestones or growth curves – correction for some confounding factors could be achieved by customisation – for example - gender, age, ethnicity, population of reference – could these be used in clinical education to generate ‘growth of learner performance’ curves. Sub-analysis may lead to greater understanding of the non-thriving learners - the “under effort for time” learner versus the “under performance for effort” learner in the analogy of the skinny child versus the stunted child!

Theme 3: Faculty Development - "Cultivating Deliberate Expertise"

We considered how learning curve analysis could inform medical registration policy for re-certification if the decay curves were further developed. The unit time for performance degradation may not be identical in each cycle for revision learners. The theory of spacing learning activity to afford deeper learning over time may contribute to determining the appropriate cycle time.