04
User experience, conclusions and future focus
What our learners say about the training they followed
Looking at the combined ‘agree’ and ‘strongly agree’ responses of learners to the closed questions of the feedback provided for each session, it can be seen that 92% felt that they would be able to use what they learned in their work, 94% of respondents thought that the trainer(s) were knowledgeable about the subject matter and, overall, 90% were satisfied with the training.
To explore the open-text feedback provided, an AI large-language model (RoBERTa) trained to classify the emotional tone of texts was used to generate scores for open-text feedback, known as sentiment scores, placed on a scale of –1 to 1, with –1 being the most negative comments, 0 being neutral and 1 being the most positive comments. These sentiment scores show moderate to strong correlations with the satisfaction measures obtained from the closed questions, which supports their validity. However, they also capture aspects of user feedback not covered solely by the closed questions. This is shown in the analysis using a quadrant plot.
Learner feedback on training sessions
Source: European Union Agency for Asylum (EUAA)
The quadrant plot illustrates the relationship between sentiment (horizontal axis) and satisfaction (vertical axis). Each point in the plot represents a training session, and both axes are dichotomised at their median values.
The top-left quadrant reveals unexpected negative comments (negative sentiment with high satisfaction), whereas the bottom-right quadrant captures unexpected positive comments (positive sentiment but low satisfaction). In contrast, the bottom-left and top-right quadrants show alignment between emotional tone and satisfaction levels. These findings underscore that open-text feedback does not always align with satisfaction metrics, providing valuable input for targeted improvements.
Relationship between emotional tone of comments and satisfaction ratings
Word clouds showing frequently used words in negative (left cloud) and positive (right cloud) user comments
To explore the open-text content further, a quantitative analysis of user feedback was carried out, capturing the most frequent adjectives used to describe the training. These are plotted using word clouds where the size of each word is proportional to its frequency in the original texts. The left-hand plot visualises negative comments (low sentiment, i.e. ≤ -0.5), which predominantly revolve around online delivery, technical difficulties, module complexity and module duration, while the right-hand plot shows positive comments (high sentiment, i.e. ≥ 0.5) and underscores the training’s usefulness, quality and overall satisfaction.
In general, the open-text feedback shows a clear overall positive trend, with our RoBERTa-based sentiment analysis model indicating that 4 out of 10 responses are ‘very positive’ and another 2 out of 10 are ‘positive’. Only 2 out of 10 are neutral, and the remaining 2 out of 10 are divided evenly between ‘negative’ and ‘very negative’ . This distribution suggests that the majority of users were satisfied with the training, though some key issues emerged from those with less favourable experiences.
Using an advanced AI method to extract topics (BERTopic), we observed distinct themes in both positive and negative comments. Those leaving very positive feedback most often praised the overall course quality (around 22% of all positive mentions), trainers’ expertise (11%), and the clarity or usefulness of the materials (29%). In contrast, for the very negative group, the biggest concerns centred on translation difficulties (14%), IT issues (11%), material and content issues (11%), training format / online delivery (10%), assessment (9%), and trainers/facilitators (3%).
Two tables of comments for the top 10 and bottom 10 sessions ranked by the satisfaction measure, derived from the closed questions, were generated and then analysed. From the 10 sessions with the highest satisfaction, the comments tended to concern achievement of the objectives, syllabus content, dynamic style of presentation and application to their work, interspersed with a few cautionary remarks and areas to consider for improvement. The feedback from these various training sessions (from the ‘Tailor-made misc.’, ‘Trainer’, ‘Reception’, ‘Caseworker’ and ‘Manager’ modules) reflected a high level of satisfaction, with scores consistently above 95%. Participants appreciated the clear objectives, the practical and accessible content, and the dynamic methodologies employed by trainers, which facilitated both learning and engagement. Many highlighted the usefulness of the training in addressing specific skills, such as public speaking and the application of theoretical knowledge in practical scenarios. The online components were praised for their user-friendliness and interactivity, while face-to-face sessions were valued for their structure and opportunity for discussion. Suggestions for improvement included extending the duration of certain training modules and incorporating more role-playing activities. Overall, the sessions were well-organised, met or exceeded expectations, and provided valuable tools and knowledge for participants’ professional development.
From the 10 sessions with the lowest satisfaction, comments related to level of difficulty (too hard or too easy), problems with completing training during working hours, material not targeted to the learner group, difficulties with navigation and language. The open-text feedback from these lowest-scoring sessions, i.e. below 60% satisfaction (from the ‘Foundation’, ‘Interpretation’ and ‘Manager’ modules), indicated several challenges and areas for improvement. Participants reported that the material was often too extensive, not always relevant to their roles, and sometimes difficult to navigate or follow, especially in different language versions. Technical issues, such as complicated navigation and the inability to skip content, caused frustration. Some found it difficult to participate during working hours, and the lack of translation during certain sessions hindered understanding. While some appreciated the usefulness of the material and the quality of exercises, others felt the content and design needed updating, with suggestions to include more engaging elements like videos and animations, and to reduce the theoretical load. Overall, the sessions received lower satisfaction scores, reflecting the need for clearer, more concise and better-adapted training materials.
A further two tables of the top 10 and bottom 10 comments ranked by the sentiment score derived from the open-text feedback were analysed. As the sentiment scores capture the emotional content of texts, they tend to indicate enthusiasm for or dislike of the corresponding session, but they also provide some more detail about why learners felt this way. The top 10 comments tended to concern liking the session, learning a lot, feeling satisfied, looking forward to using the material in their work and wanting to attend more training in the future. The feedback for these 10 sessions across various training modules (‘Reception’, ‘Foundation’, ‘Tailor-made misc.’, ‘Manager’, ‘Trainer horizontal’ and ‘Vulnerability’) highlights overwhelmingly positive experiences, showing sentiment scores above 0.986 on a scale of -1 to 1, with participants expressing satisfaction, enthusiasm and gratitude for the content and delivery of the sessions. Attendees appreciated the engaging presentations, practical knowledge, and opportunities to apply newly learned techniques in their work. Many praised the trainers’ expertise and expressed eagerness to join future sessions, describing the training as highly valuable, well-organised and impactful. The specific modules of ‘Reception’, ‘Tailor-made misc.’ and ‘Trainer Horizontal’ received commendations for exceeding expectations and fostering meaningful connections among colleagues. Overall, participants view these training modules as enriching opportunities for professional growth and skill enhancement.
In contrast, the 10 lowest comments relate to lagging video and navigation problems, criticism of design, level of difficulty (too easy), material not targeted to the learner group and use of role-play. Participants in these 10 sessions reported significant dissatisfaction with several training modules (‘Registration’, ‘Foundation’, ‘Vulnerability’ and ‘Caseworker’), showing sentiment scores of below -0.908 and citing persistent technical issues such as videos that lagged or failed to load, which forced some to guess answers and left others unable to complete the course. Many described the course design as poor, with serious mistakes in case studies, unnecessarily complex navigation, and a lack of clear instructions. Some found the content repetitive, lacking in depth, or irrelevant to their professional needs, with one participant stating that the training was more suitable for elementary students than police officers. Overall, these sessions were perceived as frustrating, unhelpful and below the expected standards, leading some to state that they would not participate again.
These findings offer practical insights into areas where the training excels, particularly the comprehensive content and dedicated facilitators, and where improvements could further enhance user satisfaction, such as refining online delivery methods and ensuring more robust IT support.
How our vocational training is linked to job tasks
To supplement the materials derived from the feedback forms following training sessions, the Agency also contacted its alumni in three Member States (Portugal, Spain and Sweden) to gather testimonials on their experiences of training. Training national contact points, managers, caseworkers and trainers provided short descriptions, highlighting their experiences as well as how EUAA training relates to the job tasks of officials in their settings. These can nicely be captured in the statement ‘The training of the EUAA has been key, providing us with a space for reflection, exchange and learning that has allowed us to place international protection as a central aspect in our professional work” (Paula Barros, Childhood and Migration Service (Training Focal Point, Madrid)).
Nine testimonials, covering a variety of reflections, are provided below.
Training Plan and catalogue news
Overview of the EUAA Training Plan 2024
The EUAA Training Plan, available on the official EUAA website, provides a structured and systematic approach to training delivery. These plans cover the training curriculum in the context of permanent support organised by the EUAA. To further enhance accessibility, in 2024, a dedicated plan was introduced to outline all EUAA tailor-made training sessions, providing stakeholders with a comprehensive overview of available training opportunities. The plans include information on training plan, name of the module, target group, location, estimated time for online studies, registration deadline, online duration, webinar / face-to-face dates and assessment date.
In response to insights gathered from the Training Needs Survey, in 2024 the EUAA increased its focus on webinars by nearly 10% compared to 2023 as a key component of face-to-face training delivery. This enhancement has reduced the need for physical presence, making participation more accessible for learners across Member States.
Training Catalogue 2024
The Training Catalogue provides an introduction to the European Asylum Curriculum and then presents each module, which is placed into one of the following categories: Advanced core modules, Country of origin modules, Dublin modules, Exclusion modules, Modules for managers, Reception, Registration, Resettlement, Vulnerability, Communication and information provision, Qualification for international protection, and Orientation course. The modules have been developed within the framework of the Common European Asylum System’s (CEAS) legal instrument – the Convention Relating to the Status of Refugees (the Refugee Convention) of 1951 and its Protocol – and other relevant international and European law.
In 2024, the EUAA launched five new or upgraded training modules. The newly developed registration modules are designed for officials who first come into contact with individuals who may need or wish to apply for international protection, as well as those responsible for registering and lodging applications. These modules form the ‘Registration modules’ component within the EUAA Training Curriculum. Additionally, the introductory and intermediate modules on the reception of vulnerable persons were revised, and an upgraded version was introduced to align with accreditation requirements and standards.
The EUAA also launched the ‘Public speaking and persuasive communication’ module to enhance the communication and persuasion skills of trainers and asylum/reception officials. This module is currently classified under the ‘Tailor-made training’ category.
To support the implementation of the new Pact on Migration and Asylum, a series of tailor-made training modules was launched in 2024. These new modules provide essential guidance on key legislative changes, helping policymakers, legal advisors and operational staff navigate the evolving EU asylum and migration framework. All new training are webinars that take between 3.5 and 6 hours to complete.
In 2024, the EUAA continued to support the translation of European Asylum Curriculum training modules, achieving significant milestones in multilingual accessibility. This increase in the number of supported languages highlights the Agency’s commitment to inclusivity and the broader dissemination of its translated resources. The consistent growth in translations over the years reflects a dedicated effort to enhance accessibility and address the diverse linguistic needs of its stakeholders.
Translation achievements in 2024:
73 translations
completed
into 17 languages
18 027 pages translated, including 2 207 pages
in non-EU languages
Nearly 12 hours
of subtitles created
to support
multilingual access
Source: EUAA
In 2024, the horizontal curriculum played a key role in training of a growing number of asylum and reception officers, alongside an expanding pool of 205 new trainers and assessors under the ‘Becoming an EUAA trainer and assessor’ module.
Additionally, interest in coaching fundamentals recorded a significant rise. The ‘Introduction to coaching’ module was delivered six times as part of a collaborative effort under the permanent and operational support initiative, increasing by more than half compared to the previous year.
The EUAA Conference for Trainers took place on 13 and 14 June 2024 in Malta, providing a dynamic platform for team building and professional exchange. Trainers from various Member States gathered to discuss and share insights on training and learning in contemporary contexts. The 2024 conference focused on skill enhancement through dedicated sessions on:
-
public speaking techniques;
-
effective feedback strategies to improve learner engagement; and
-
methods to enhance trainer effectiveness.
© European Union Agency for Asylum (EUAA)
An interactive session explored the interpretation and implementation of work-based learning across Member States, fostering creative brainstorming and collaboration. A key highlight of the event was the opportunity to strengthen professional networks among trainees from different Member States, reinforcing the EUAA’s commitment to promoting excellence in training delivery and the professional development of its network.
Plagiarism check
Academic integrity remains a priority for the EUAA. In 2024, all newly developed training modules underwent plagiarism checks to ensure high standards of originality, credibility and accuracy. This process not only identified potential instances of copied content but also evaluated the reliability of sources, ensuring that they were peer-reviewed and correctly cited. By implementing these measures, the EUAA guarantees that its training materials are trustworthy, accurate, and uphold the highest standards of academic integrity.
As part of its ongoing commitment to ethical learning, the EUAA is developing a plagiarism check tool for trainees. While still in progress, this initiative aims to help trainees verify the originality of their work and ensure proper referencing. By integrating such measures, the EUAA fosters a culture of transparency, accountability, and respect for intellectual property, providing valuable resources for both learners and trainers to engage in authentic and responsible learning.
LMS enhancements
As part of its commitment to supporting learners, the EUAA introduced the ‘Orientation for learners’ module in early 2024. This fully automated, self-paced module is designed to help both new and existing learners engage more effectively with the European Asylum Curriculum and its associated training processes. The key features of the module are as follows:
-
Guidance on the EUAA training organisation, delivery, assessment and quality assurance.
-
Information on addressing specific issues or topics within the training context.
-
Details on available technical, administrative and academic support.
With an estimated completion time of approximately 65 minutes, this module ensures easy access to essential training resources, empowering learners to navigate the EUAA training system efficiently.
Data and reporting: between 1 January and 9 October 2024, the centre responded to 224 training data requests, covering areas such as country and content-specific factsheets, training feedback and operational training data. The average delivery time was 4.2 days. Detailed monitoring reports for quarterly satisfaction surveys were also developed, leading to time and cost savings.
Training Needs Analysis: 2024 was particularly active for TNA activities, with comprehensive analyses conducted in Italy and the Western Balkans to address regional training needs. A revision of the TNA methodology was completed, introducing a TNA Fast Track procedure currently awaiting approval. Additionally, work began on an Overarching TNA approach designed for systematic application across multiple countries. This new framework aims to harmonise and integrate TNA processes across regions, supported by external expertise. These developments are expected to enhance the effectiveness and consistency of Training Needs Analyses across different regions.
Evaluation and accreditation activities: the Centre played a key role in supporting the Agency’s evaluation function by coordinating evaluations of operational plans and roadmaps while monitoring permanent support activities. Activities also included contributing to licensing and accreditation efforts for the EUAA Academy programme and developing measurement tools for the EUAA Monitoring Project, which is set to launch in January 2025. Additionally, the Centre provided analytical support for a quasi-experimental study using its Quality Assurance Tool (QAT) scores to inform randomised control trial planning. It also actively refined methodologies for creating the Annual Training Plan by reviewing existing procedures to enhance efficiency and ensure robust future planning processes.
Conclusions and future focus
The EUAA made significant strides in addressing the growing demand for asylum training across the EU in 2024, with key achievements including a substantial increase in training delivery, the introduction of internationally recognised qualifications, and the development of a more targeted approach through the revised Training Needs Analysis (TNA) methodology.
As highlighted in Chapter 2, the expansion of the European Asylum Curriculum, coupled with the introduction of new trainers and the overwhelmingly positive feedback from participants, further underscores the relevance and quality of the EUAA’s training initiatives. Notably, 2024 saw a record number of over 100 000 officials trained across all training activities offered, resulting in a 49% increase in operational support activities and a 13% increase in participation in permanent support activities compared to 2023. The overall satisfaction rate of 89% among participants demonstrates that the EUAA’s training activities are well-received and effective in building the capacity of asylum and reception officials.
The analyses presented in Chapter 3 provide valuable insights into the characteristics of the training programmes and the factors that influence training outcomes. The findings suggest that modules with longer durations tend to have lower completion rates, and that fully online sessions exhibit lower completion rates compared to face-to-face or blended methods. Additionally, the importance of language is highlighted, with sessions conducted in English showing marginally higher completion rates. Furthermore, sessions in the context of permanent support demonstrate higher completion rates, while EUAA-organised sessions typically exhibit slightly lower completion rates. In terms of trends in content themes, foundation modules took the lead in 2024, accounting for 33.60% of all participations. Overall, the EUAA’s training programmes are effective in building the capacity of asylum and reception officials, and its training activities are well-aligned with the needs of its stakeholders.
The feedback from learners, as presented in Chapter 4, highlights the overall satisfaction with the EUAA’s training initiatives, with 92% of respondents feeling that they would be able to use what they learned in their work, and 94% of respondents thinking that the trainers were knowledgeable about the subject matter. The analysis of open-text feedback, using a large-language model, revealed moderate to strong correlations with the satisfaction measures, and provided valuable insights into the aspects of the training that learners found most useful, such as the practical application of the training content, the quality of the trainers, and the relevance of the training to their work. The quadrant plot analysis showed that while there was a general alignment between sentiment and satisfaction, there were also instances of unexpected positive or negative comments, highlighting the importance of considering both quantitative and qualitative feedback in evaluating the effectiveness of the training. Overall, the learner feedback suggests that the EUAA’s training initiatives are having a positive impact on the knowledge, skills and competencies of asylum and reception officials, and provides a solid foundation for the Agency’s future training activities.
Looking ahead, the EUAA is committed to expanding its training offerings to meet the evolving needs of Member States, with plans to introduce additional accredited modules to support Member States in preparing for the Pact’s legal requirements, while also providing opportunities to earn internationally recognised qualifications. This also includes continued support to Member States experiencing migratory pressure, with the aim of helping strengthen the overall capacity for asylum and reception system. The launch of a European Master’s in Asylum and Reception Management will further contribute to this goal, ensuring that asylum and reception officials are well-equipped to manage asylum procedures in line with EU standards, strengthening the professionalisation of the sector and contributing to greater convergence across EU asylum systems. By continuing to build on its successes and address the evolving needs of its stakeholders, the EUAA is well-positioned to lead the transformation of the asylum and reception sector, with a strong focus on professionalisation, quality and impact.