From Training Needs Analysis to Evidence-Based Performance Systems: Rethinking TNA in CBTA and EBT Contexts

Training Needs Analysis (TNA) represents one of the conceptual and methodological pillars of training and development systems within complex socio-technical environments. It can be understood as the structured process through which discrepancies between actual and expected performance are identified and interpreted, with the aim of determining whether, and to what extent, training interventions are necessary, justified, and effective.

In the classical literature, TNA is defined as a systematic analysis of training needs intended to support the design of interventions aligned with organisational objectives and with the characteristics of tasks and individuals (Goldstein & Ford, 2002). A more in-depth reading, however, highlights that TNA cannot be reduced to a mere preliminary phase of the training cycle; rather, it should be conceptualised as a complex and systematic analytical device, situated at the intersection of performance theory, work psychology, and the dynamics of socio-technical systems (Hollnagel, 2014).

The fundamental assumption underlying TNA is the concept of the “performance gap”, namely the difference between what individuals actually do and what they are expected to do. However, this discrepancy does not automatically imply the need for training, as the causes of inadequate performance may be multiple and include organisational, technological, procedural, or motivational factors. This makes a rigorous distinction between competence-related issues and system-related issues essential (Salas et al., 2012). In this sense, TNA assumes a critical function, preventing a simplistic view of training as a universal solution and instead contributing to a more sophisticated analysis of performance determinants.

The classical model proposed by Goldstein and Ford (2002), articulated across the three levels of organisational, task, and individual analysis, continues to represent a fundamental reference for structuring TNA. However, it reflects a relatively static and linear perspective, in which training needs are identified in an initial phase and subsequently translated into predefined training programs. This approach has been progressively challenged by the evolution of operational contexts, which are increasingly characterised by high levels of complexity, interdependence, and variability, thus requiring more dynamic and adaptive approaches (Noe, 2020). This model implies a relatively rigid logical sequence, in which needs analysis precedes and directly determines training design, assuming that operational conditions and performance requirements remain stable over time. Within this perspective, the relationship between identified needs and training solutions appears linear and predictable, leaving limited room for iterative revision or in-process adaptation. However, in complex contexts, such linearity becomes difficult to sustain, as needs tend to emerge and evolve dynamically through the interaction between actors, technologies, and the operational environment.

Within this evolutionary trajectory, contributions such as that of Casey and Doverspike (2005), “Training Needs Analysis and Evaluation for New Technologies Through the Use of Problem-Based Inquiry”, anticipate a less sequential and more exploratory view of TNA, emphasising the role of problem-based inquiry in the identification of training needs. In particular, the authors highlight that “training needs are best understood when embedded within real problem contexts rather than abstract skill deficiencies” (Lewis & Smith, 2005, p. 115), suggesting that understanding training needs requires a situated analysis capable of capturing the complexity of operational contexts and emerging technologies. From this perspective, TNA is no longer limited to a process of gap identification, but becomes a problem-oriented investigative activity, in which learning emerges from the interaction between actors and context, reinforcing the idea that “effective needs assessment must be grounded in authentic performance challenges” (Lewis & Smith, 2005, p. 118).

This orientation is particularly relevant in contexts characterised by rapid technological innovation, where needs cannot be fully anticipated but instead emerge through the interaction between operators, tools, and the environment, reinforcing the idea that TNA must evolve toward more dynamic, iterative, and context-sensitive models.

Within this framework, the domain of commercial aviation represents a particularly significant case, where the introduction of Competency-Based Training and Assessment (CBTA) and Evidence-Based Training (EBT) has profoundly redefined the role and nature of TNA, without explicitly referring to the term itself. It is noteworthy that the International Civil Aviation Organization Doc 9995 Evidence-Based Training Manual (ICAO, 2013) does not explicitly refer to TNA in the classical sense. This terminological absence, however, does not indicate a marginalisation of the concept, but rather its transformation into an implicit systemic function distributed within the EBT cycle. Conceptually, all the constitutive elements of TNA are present in Doc 9995 in the form of processes for identifying training priorities based on operational data, performance analysis, scenario selection, and continuous competency monitoring, thus configuring an evolved TNA, no longer conceived as a discrete phase but as an integrated component of a dynamic system.

This transition represents a significant epistemological shift, which can be described as a move from a training-centric perspective toward a systemic, performance- and safety-driven approach, in which training is no longer the starting point of analysis but rather the outcome of a continuous process of interpreting operational data and performance dynamics. Accordingly, “training needs” are no longer defined in static terms, but emerge as a dynamic property of the system, observable through performance patterns and operational risk indicators.

Within the CBTA framework, formalized in ICAO Doc 9868 PANS-TRG, TNA is implicitly embedded in the definition of competency frameworks and in the design of training programs based on observable behaviours, performance criteria, and proficiency levels, thereby moving beyond the traditional logic cantered on training hours and toward a qualitative and context-sensitive evaluation of performance (ICAO, 2022). In this sense, TNA evolves into a continuous competency gap analysis, in which the focus shifts from the mere identification of deficiencies to the understanding of the conditions under which specific competencies are generated, activated, degraded, or adapted.

Within the EBT paradigm, this evolution is further reinforced, as the identification of training needs derives directly from the analysis of operational data such as Flight Data Monitoring (FDM), Line Operations Safety Audit (LOSA), Air Safety Reports (ASR), and training evaluation results (including full-flight simulators), making TNA an inherently data-driven process closely integrated with safety management systems (ICAO, 2013).

In this context, training needs are no longer defined a priori, but emerge from the observation of recurrent error patterns, operational vulnerabilities, and criticalities in threat management, enabling a risk-based prioritisation grounded in their potential impact on safety. This approach reflects a view of human performance consistent with the principles of Safety-II, according to which performance variability should not be interpreted solely as negative deviation, but as an expression of the adaptive capacity of operators within complex systems (Hollnagel, 2014). Consequently, TNA is no longer limited to identifying deficits but can be reinterpreted as a tool to understand and support operational resilience, by analysing both conditions of failure and success. As Hollnagel (2014) emphasises, performance variability constitutes the basis of both successes and failures in complex systems, suggesting that needs analysis must necessarily include an understanding of how performance succeeds under variable conditions.

Within this perspective, the transformation of TNA entails significant theoretical and practical implications. On the one hand, it implies the abandonment of linear and sequential models in favour of circular and iterative approaches, in which needs analysis, training design, competency assessment, and operational feedback continuously inform one another within a closed-loop system. On the other hand, it requires an increasingly tight integration between operational data, training systems, and decision-making processes, thus making TNA a distributed and continuous capability rather than an episodic function.

Despite these significant and transformative developments over time, important challenges remain, including the complexity of data interpretation, the risk of information overload, and the need to develop adequate analytical capabilities within complex organizations.

Moreover, the absence of explicit terminology, as in the case of Doc 9995, may generate interpretative ambiguities, making it necessary to undertake a theoretical effort to reconcile traditional models with emerging practices.

From this perspective, TNA should be understood as a fundamental diagnostic function within the instructional design process, as it allows for determining whether a performance discrepancy is actually attributable to deficits in knowledge, skills, or competencies, or whether it stems from organizational, procedural, technological, or managerial factors. For this reason, effective TNA requires adequate analytical and methodological competencies on the part of those who conduct it. Its value does not lie in automatically legitimizing a training intervention, but in supporting a grounded diagnosis of the nature of the problem. In this sense, TNA does not represent a panacea for organizational issues, but rather a critical tool for distinguishing when training is truly necessary and when non-training interventions are required.

In conclusion, while maintaining its roots in classical models based on the performance gap (Goldstein & Ford, 2002), TNA has evolved into a systemic and dynamic process. In contemporary contexts such as CBTA and EBT, it is embedded within a broader architecture of performance and safety management, moving beyond an exclusively training-centric perspective. Within this framework, TNA does not disappear as an analytical function, but is dynamically integrated into organizational systems, contributing to performance diagnosis and to the distinction between training needs and system-related issues (Salas et al., 2012; Noe, 2020; Hollnagel, 2014; ICAO, 2013; ICAO, 2022).

References

- Goldstein, I. L., & Ford, J. K. (2002). Training in organizations: Needs assessment, development, and evaluation (4th ed.). Wadsworth.

- Casey, M. S., & Doverspike, D. (2005). Training needs analysis and evaluation for new technologies through the use of problem-based inquiry. Performance Improvement Quarterly, 18(1), 110–124. https://doi.org/10.1111/j.1937-8327.2005.tb00328.x

- Hollnagel, E. (2014). Safety-I and Safety-II: The past and future of safety management. Ashgate.

- International Civil Aviation Organization. (2013). Doc 9995: Manual of Evidence-Based Training. ICAO.

- International Civil Aviation Organization. (2022). Doc 9868: Procedures for Air Navigation Services (PANS-TRG). ICAO.

- Lewis, L. H., & Smith, D. H. (2005). Training needs analysis and evaluation for new technologies through the use of problem-based inquiry. Performance Improvement Quarterly, 18(1), 110–124.

- Noe, R. A. (2020). Employee training and development (8th ed.). McGraw-Hill.

- Salas, E., Tannenbaum, S. I., Kraiger, K., & Smith-Jentsch, K. A. (2012). The science of training and development in organizations: What matters in practice. Psychological Science in the Public Interest, 13(2), 74–101.

- Dick, W., & Carey, L. M. (1977). Needs assessment and instructional design. Educational Technology, 17(11), 53–59.

- Dick W., & Carey L. (1990). The Systematic Design of Instruction. Harper Collins.