81318914 - Consultancy on Monitoring & Evaluation

artificial intelligence act: technical assistance for ai safety - These lots all share a similar tasks and services: - The organisation of multiple risk modelling workshops together with the AI Office, producing a risk model, risk scenarios, as well as corresponding thresholds, informing further work on evaluations. - The development of newly evaluation tools that are private to the Commission. - The onboarding of existing evaluations that are publicly available into the developed risk framework. - The creation of a reference procedure and reporting template for doing risk assessment with respect to the developed evaluations. - Ad hoc and on demand services for complex evaluations, including for example red-teaming and uplift studies. - A risk monitoring service in the form of regular briefings on new developments (e.g. models, risk sources, elicitation techniques, mitigation, incidents) to complement existing AI Office monitoring actions. The scope of these tasks varies by lot. Furthermore, there is one lot that is cross-cutting across various risk. - Lot 6: agentic evaluation interface, procuring software, Cloud infrastructure support, and methodology to evaluate GPAI on diverse types of benchmarks, focusing on agentic interaction and complex agentic benchmarks.