EECA Regional SORT IT Going virtual for research training during the COVID-19 pandemic and beyond: e-SORT IT

The Structured Operational Research and Training Initiative (SORT IT) model has contributed to building research capacity and has produced evidence for improving public health program performance in countries with limited research capacity. The model involves hands-on mentorship and consists of three modules/weeks. It is recognized to be an innovative research capacity building model. In a world changed by COVID-19, where bringing people together is not viable, an innovative, interactive, web-based, knowledge-transfer platform (e-SORT IT) for virtual implementation of SORT IT modules was created. The platform design imitated the residential course as closely as possible with the same lectures, plenary sessions, and breakout rooms. Despite the challenges, the platform performed well and even though participants and mentors were located in eight different time zones, the course was successful; 90% of participants achieved their milestones and 10 manuscripts were successfully completed. Participant evaluation revealed a satisfaction level that was nearly equivalent to the residential module. However, mentor evaluation indicated a number of shortcomings including capacity building, professional networking, communication, engagement, and contribution by participants, as well as overall module success. In conclusion, COVID-19 stimulated the creation of the e-SORT IT platform that provided a functional alternative to the residential version. Despite the limitations of reduced capacity building and networking, the e-SORT IT platform should be considered a success – it delivered the goods. This is an example of innovation and flexibility, two attributes that are sorely needed to maintain activities during the pandemic and beyond


The Structured Operational Research and Training Initiative (SORT IT)
Evidence generated through operational research is key for evidence-based decision making and improved performance of health programs. However, not all countries have enough research capacity to ensure the conduct of high-quality research. This capacity is lowest in Low and Middle-Income Countries, where research evidence is highly relevant and most needed. In such countries, the Structured Operational Research and Training Initiative (SORT IT) model has contributed to building research capacity and has produced evidence for improving public health program performance [1][2][3][4][5]. The model involves hands-on mentorship and consists of three modules (one week each for protocol development, data handling, and write up). The third module (write up) is conducted about sixeight months after completion of the first two modules to allow the time needed for data collection and analysis [6]. An independent evaluation of SORT IT recognized the model as being of high value and an innovative way of research capacity building. It recommended applying this model to other areas of health research. [7]. Ongoing evaluations of SORT IT alumni eighteen months after course completion, moreover, show that nearly half of them engage in further research projects after the course and a substantial proportion of those are published, illustrating that the acquired research skills 4S are used independently [8][9][10]. However, face-to-face (residential) modules involve considerable logistics and costs for coordination, travel, and accommodation. Bringing people together from different parts of the world during a pandemic such as Coronavirus disease 2019 (COVID-19) or in conflict settings is also not practically possible. An alternative way of coping with the high demand for training is to offer SORT IT online video lectures for those interested [11]. The effectiveness of the online lectures was demonstrated by a first published paper that piloted the experience during the conduct of a National Armenian e-SORT IT program which relied on these online resources [12][13][14]. In a world changed by COVID-19, this experience stimulated the development of an innovative, interactive, web-based, knowledge-transfer platform (e-SORT IT) for virtual implementation of SORT IT modules. The Tuberculosis Research and Prevention Center (TBRPC) developed the platform and made it available for SORT IT courses during the COVID-19 pandemic with the support of the UNICEF, UNDP, World Bank, WHO Special Programme for Research and Training in Tropical Disease (TDR).

Online e-SORT IT
Initially, the main purpose of the e-SORT IT was to have a tool that allowed knowledge transfer to those who applied to SORT IT courses but could not be selected due to the restricted number of available places. The online e-SORT IT platform facilitates conducting the SORT IT course while preserving its structure and milestones. As such, it is viable alternative for those who are unable to make it to the residential training for any reason. Using the online platform, practical activities and online video lectures are organized in a way that allows the same step-by-step development of study protocols and manuscripts as in a face-to-face course. It allows participants/students to dedicate time and opportunity to participate in the training ('on the job') with interactive discussions with mentors. The platform is also a complementary tool to improve the communication between course trainees and mentors, facilitate sharing of training materials and assisting in the overall organization of the course. The interactive, web-based e-SORT IT platform has the functions presented in the Figure 1.

Going virtual during a pandemic and beyond: e-SORT IT
In 2020, the COVID-19 pandemic resulted in severe travel restrictions around the globe which made it impossible to conduct classical residential SORT IT courses. In January 2020, a SORT IT course in Ukraine had just completed its first two modules using the residential model when the pandemic struck. This made it impossible to conduct the third, write up module, planned for June 2020 due to lockdowns and travel restrictions. Given this crisis, the online SORT IT platform was then repurposed and adapted to facilitate virtual conduct of the third module. The platform design imitated the residential course as closely as possible with the same lectures, plenary sessions, and breakout rooms as in the residential course using a distance video communication platform. All participants and mentors were trained to use the platform and it was found to be quite intuitive. Despite the challenges, the platform performed well and even though participants and mentors were located in eight different time zones, the course was success; 90% (9 out of 10) of participants achieved their milestones and nine manuscripts were successfully completed. Following Module 3, an evaluation including participants and mentors was performed. An online survey was conducted using Likert 5-point scale (Very poor, Insufficient, Average, Good, Excellent) to evaluate overall success and several other aspects of the module. Results of the survey are presented below in Figure 2. Percentages in the figure are calculated by dividing the total sum of the scores of the question by the highest possible result of that question. 5S Participant evaluation revealed a satisfaction level that was nearly equivalent to the residential module. However, mentor evaluation indicated a number of shortcomings including reduced capacity building, professional networking, communication, engagement, and contribution by participants, as well as overall module success. Only access to course materials was slightly improved.
The advantages of virtual or online courses are evident in terms of cost-savings from international travel and related expenses, as well as from improved access to the SORT IT materials. However, this comes at the expense of some aspects of capacity building and engagement with participants that are the cornerstones of the SORT IT courses. As virtual and hybrid (combination of virtual and residential) trainings may be the only possible ways to conduct SORT IT in the near future, solutions need to be sought to cover the identified shortcomings. The virtual mode of SORT IT training is novel, and will likely benefit from continuous innovation and modifications. Reduction of continuous online hours per day while increasing overall training duration, introduction of new engagement techniques and specific time zone adaptations of the schedule have the potential to improve this type of training.
In conclusion, COVID-19 stimulated the creation of the e-SORT IT platform that provided a functional, albeit less desirable, alternative to the residential version. Despite the limitations of reduced capacity building and networking, the e-SORT IT platform should be considered a success -it delivered the goods. This is an example of innovation and flexibility, two attributes that are sorely needed to maintain activities during the pandemic and beyond.

Acknowledgements and funding
TDR and partners can conduct their work thanks to the commitment and support from a variety of funders. These include our long-term core contributors from national governments and international institutions, as well as designated funding for specific projects within our current priorities. A full list of TDR donors is available on our website at: https://www.who.int/tdr/about/funding/en/. We are grateful to all these donors and particularly those who have supported research capacity building activities which allow health workers to save lives on the frontlines. The SORT IT virtual platform development and adaptation for East Europe and Central Asian countries was funded by TDR and USAID.