The system of conducting examinations of grant projects for "Inconsult K"

The project

Since 2010, the company has been acting as an operator of two major government programs, providing their organizational, technical, informational and analytical support: complex cooperative projects to create high-tech production (Resolution No. 218) and the mega grant program uniting the world's leading scientists (Resolution No. 220). Among the customers of the services are the Ministry of Science and Higher Education of the Russian Federation, the Federal Agency Rossotrudnichestvo and a number of large technology corporations.

4,200 experts from 30 countries are involved in the examination of competitive initiatives. To work with them, Inconsult K has been using the system for conducting expertise for more than 10 years, which is owned by the Directorate of Scientific and Technical Programs, which is engaged in the implementation of federal targeted programs. Over time, it became less and less convenient to use the system: the functionality did not allow us to fully solve the company's tasks, and it was not possible to change someone else's system. The company saw the optimal solution in the development of its own system.

Tasks

The new system was supposed to solve the following tasks.

Gather all the experts
Until now, instead of a single standardized list, disparate Excel tables have been used. It was necessary to create a unified expert environment that allows you to interact with experts, evaluate their work and experience of interacting with them. And the first step on this path is the creation of a consolidated register of experts.

Speed up the examination processes
Automation of most typical processes would free up tens of hours of curators' and experts' working time and ensure the best quality of expertise. In particular, to allow a comparative analysis of applications, to compile summary conclusions based on private expert opinions.

Create a single workspace
All objects of expertise, accompanying documents, examination tasks, private expert opinions and summary examination based on them were to be concentrated in a single information system with Internet access and differentiation of user rights.

Simplify the work of curators
The curators responsible for the timeliness and quality of examinations had to receive a flexible and effective tool for managing examinations: the ability to set deadlines, regulations, invite relevant experts and evaluate their work, monitor the performance of work. It was also necessary to provide rapid training for new users so that they could get involved in the work as quickly as possible.

Implementation

Our solution TOT Expertise was optimally suited to the customer's requirements: modern adaptive design, the ability to flexibly customize forms, regulations and roles, tools for comprehensive assessment and the formation of summary expert opinions. The main task — facilitating and speeding up the work of experts — is solved by our system by default. All materials for the examination (objects, assessment methodology, documents) are collected in one web form.

We deployed a bilingual examination system, transferred data from disparate tables to the unified register of experts, configured the import of applications from expert campaigns launched at that time to TOT Expertise.

Experts are busy people, their time is too expensive to spend it studying voluminous printed instructions. Therefore, we packed the training to work with the system into a series of short, capacious videos.

We started working on the project in July 2020, and already in August the first expert companies began to carry out the system. On the part of Inconsult K, the project was handled by the deputy director, the head of the expertise department and a group of expertise curators. From ours — analysts, developers and testers, almost the entire team.

Results

What the customer got

1. Own system of expertise. A flexible and modern platform that he can customize at his discretion without resorting to the help of developers. Expert campaigns can be carried out using ready-made templates or customize any of their characteristics: fields in questionnaires, user roles, the life cycle of the task for examination and deadlines for completing tasks, automatic notifications. One employee can now launch such a campaign, which includes thousands of objects and involves hundreds of experts, in almost a few clicks. All communication between users takes place inside the system, they receive automatic email notifications about every event related to their work. Here, the customer can conclude contracts with experts who responded to the invitation.
2. Unified Register of Experts. A detailed list of all experts who have ever been invited and participated in expert campaigns, including data on scientific publications, the affiliation of the expert, all his conclusions, comments and assessments of his work by the curator. Experts are ranked on the basis of assessments of their work, including the quality and speed of the examination and the frequency of refusals to participate in the examination. When creating a new expert campaign, the system automatically selects relevant experts with the highest rating and sends them automatic invitations. Each expert now receives an automatic invitation to participate in a relevant expert campaign in Russian or English.
3. Acceleration of examination processes. Thanks to the automation of the processes of organizing the campaign, convening experts, collecting opinions and forming a summary opinion, it became possible to conduct more examinations per unit of time.
4. Expert bounce log. This tool, which is extremely necessary for the customer, simply did not exist before, we created it from scratch. If an expert refuses to participate in the examination, he can justify his refusal, and this affects his rating in the system. Collecting such feedback from experts makes it possible to improve the convenience and functionality of the system, makes the rating of experts more correct, and also improves the customer's understanding of the actual timing of a particular examination.
5. A convenient tool for remote work with international experts. According to user reviews, it has become much more convenient to conduct an online examination in the system, and experts are ready to continue working in it. Some experts gave valuable comments that will be taken into account in the development of the project.

Indicators in the first month

6 campaigns
were conducted in the new system

1700 tasks
for examination were issued in the system

500 experts
took part in expertises

500 objects
passed the expertise