TU Delft Computer Science Research Assessment 2015-2020

Last year, the TU Delft computer science research programs were evaluated, comprising a reflection on the preceding six years (2015-2020), and an outlook the next six years.

The assessment follows the Strategy Evaluation Protocol (SEP), used by all Dutch universities, which focuses on research quality, societal impact, and viability. The assessment is conducted by an international external committee. It is based on a self-assessment written by us, as well as a two-day site visit by the committee.

Arie van Deursen and Alan Hanjalic in the Computer Science building

At TU Delft, computer science is organized into two departments: Intelligent Systems, chaired by Alan Hanjalic and Software Technology, chaired by me.

In 2021, Alan and I together worked hard to compile our self-assessment report. It is based on extensive discussions with and contributions by many people, both inside and outside the two departments. It contains our thoughts on what we want computer science in Delft to achieve (our mission), what we did to achieve this (our strategy), an assessment of the success of our approach (our SWOT), and a plan of action for the next years ahead (our future strategy).

We proudly make the full self-assessment report available via this link — the only modification being that for reasons of privacy we omitted some of the appendices that could be traced back too easily to individual faculty members.

As part of the protocol, the committee’s findings, as well as the reaction of the executive board to these findings, have been made available as well, at the central TU Delft site. The committee is “positive about the very high and often excellent research quality, the high quality of the staff as well as the energy, drive and potential of the primarily junior research staff of both departments,” and “recognizes the relevance and societal impact of the researchcarried out the INSY and ST departments.”

We are grateful to the external committee, and in particular for the 17 recommendations that will help us further strengthen TU Delft computer science. We have integrated these recommendations in the action plan already laid out in our self-assessment, and look forward to work with everyone in our departments and our faculty to execute this action plan in the next few years.

Below, we provide the executive summary of our self-assessment, and we invite you to have a look at our full report.

Self-Assessment Summary

The phenomena of datafication and AI-zation reflect the increasing tendency to quantify everything through data and to automate the decision-making processes that are also largely based on data. Since these phenomena have entered all segments of our lives and since research in computer science (CS) is at the heart of the technological developments underlying these phenomena, CS as a research field has gained strategic importance. TU Delft Computer Science operates at the forefront of these developments with the aim to help society at large, by enabling it to maximally benefit from these phenomena, while protecting it from potential risks. To that end, inspired and driven by the TU Delft core values of Diversity, Inclusion, Respect, Engagement, Courage and Trust (DIRECT), our mission includes (1) conducting world class research in selected computer science core areas; (2) maximizing opportunities for societal impact of our research; (3) providing rigorous, research-inspired engineering education in computer science; and (4) contributing to an international academic culture that is open, diverse and inclusive, and that offers openly available knowledge.

We are organized in two departments, Intelligent Systems and Software Technology, consisting of 5 and 6 sections respectively. Sections are small-scale organizational units, typically headed by a full or associate professor and marking a particular CS disciplinary scope. While the departments are separate units, they work closely together in research and education, and collaborate for societal impact. The convergence between the departments in terms of alignment and joint pursuit of strategic and operational goals has even become so strong over recent years that we can speak of an increasingly recognizable CS entity in Delft organizing its research into five main themes transcending the departmental and section boundaries: (1) decision support; (2) data management and analytics; (3) software systems engineering; (4) networked and distributed systems; and (5) security and privacy. The themes offer critical mass in order to achieve substantial impact, and each theme involves many researchers with various CS backgrounds and expertise.

Award-winning research in these themes achieved during 2015-2020 include a novel cross-modal (e.g., combining text and images) retrieval method based on adverserial learning; genetic algorithms for the automatic reproduction of software crashes to facilitate automated debugging; and Trustchain, a permission-less tamper-proof data structure for storing transaction records of agents with applications in digital identity. International recognition of our expertise is reflected by numerous leadership roles, e.g., as general or program chairs in numerous flagship conferences, such as AAAI, EuroGraphics, ACM/IEEE ICSE, ACM OOPSLA, ACM RecSys and ACM Multimedia. In the same time period, several staff members also received the highest (inter)national recognition in their fields, such as IEEE Fellow, membership of the Young Academy of the Royal Dutch Academy of Sciences, or the Netherlands Prize for ICT Research. Our scientific reputation also brought us into the consortia of two prestigious NWO Gravitation Projects (“NWO Zwaartekracht“) of the Dutch Research Council, Hybrid Intelligence and BRAINSCAPES – the consortia that “belong to the world top in their field of research or have the potential to do so”

To maximize societal impact, we embrace eight key sectors: transport and logistics, energy, health and well-being, safety and security, finance, online education, creative industry, and smart cities. To enable and support us in making substantial interdisciplinary impact in these sectors, we have built up expertise, a network of collaborators and societal partners, and established the necessary organizational structures. Prominent examples of our impact in these sectors include the NGS sequencing analysis pipeline we designed and implemented as part of the NIPT test, which is used routinely by hospitals in several countries; Cytosplore, a software system for interactive visual single-cell profiling of the immune system; and SocialGlass, a tool suite for integration, enrichment, and sense-making of urban data. Our close ties with society are also reflected in our strategic collaborations with socio-economical partners, such as ING, DSM, Booking.com, Adyen, Ripple, Erasmus Medical Center and Leiden University Medical Center, leading amongst other things to strategic investments in the form of three large industry-funded labs (with ING, DSM and Booking.com) setup in the assessed time period for a duration of five years. Furthermore, we have invested extensive effort in public outreach, explaining and discussing science with a broad audience, and in particular in the context of complex societal debates in the domain of AI and blockchain. Finally, we play a leading role in regional, national and European initiatives, most notably in the Dutch AI Coalition (NLAIC).

In addition to scientific excellence and strong impact in the selected societal sectors, we are committed to (a) meeting the increasing societal need for highly skilled CS experts, (b) development of human capital in our organization, leading to a new generation of international academic leaders, and (c) advancing the organization and academic culture, with the key pillars of open science, diversity and inclusion.

Regarding (a), we embraced an over 100% increase of our student population, but also aim at securing the highest possible level of their knowledge, skills and academic forming despite scaling up. Therefore, we value a close connection between research and education, and let both MSc and BSc students participate actively in our research. We also formulated an ambitious strategy, the realization of which would enable us to manage this education scale-up efficiently and effectively, leaving sufficient room to our staff for further developing scientific excellence and deploying it for societal impact. Part of this strategy is the growth of our academic staff towards 100 FTE by 2024 to meet the stabilization of the student numbers (due to numerus fixus). Between 2015 and 2020, we already achieved a net growth from 54 to 72 faculty members (+33%), with more to come in the upcoming years.

Next to BSc and MSc students, we are committed to delivering highly skilled CS experts at the PhD level. The number of PhD students grew from 105 to 165 (+57%) in the assessed time period, reflecting our ability to successfully acquire research funding in the present landscape. For our PhD students, the Graduate School defines a framework in which they can develop their skills next to conducting their thesis research. We strive towards completion of PhD theses within four years and organize our supervision, official moments of assessment, requirements on the volume and quality of the conducted research, as well as evidence of scientific impact through publications, accordingly.

Regarding (b), development of human capital: as computer science expertise is in high demand across the globe, finding strong new people as well as retaining our current staff proved highly challenging, especially given the high teaching load due to our record student intake. Therefore, acquiring, developing and retaining academic talent has been one of our most important goals. Dedicated actions, such as devising of a Development Track Plan, serve to empower each staff member to provide contribution to the organization in his/her own way, based on individual interests, talents and ambitions, and in view of our joint ambition as organization.

In view of (c), our organization, we embrace open science, with a substantial percentage (80% in 2020) of our articles available as open access, and by making numerous software tools and data sets openly available. We are a highly international organization with employees and students from all over the world. We strive to be an inclusive organization, where staff and students feel at home and valued, regardless of their background, age, gender, sexual orientation or functional disability. In terms of female faculty, we realized a net growth from 11 to 14 faculty members. As the number of men employed also increased, the percentage of female faculty stayed stable at around 20%. We consider this too low. We are committed to addressing this, for which we will take a long-term approach with, amongst other means, dedicated budget reserved for continued openings for female faculty in the upcoming years.

We are proud of our scientific successes and societal impact in the core computer science disciplines as well as in interdisciplinary research in our target societal sectors. This is especially so as those were achieved in a period that was transformational for TU Delft Computer Science, characterized by substantial growth and development across our organization and activities. We anticipate an even stronger societal demand for our research and expertise in the future. We will therefore continue to initiate, participate and take on a key role in effective and interdisciplinary partnerships at the university (TU Delft AI), regional (LDE), national (ICAI, IPN), and European (ELLIS, CLAIRE) levels. Furthermore, we will continue the growth path for our staff, in order to build up capacity enabling us to further develop our scientific excellence and offer our strongly increased student population the world-class research-intensive education they deserve. To achieve this, we center the next steps in our ongoing transformation around people, organization, and profiling and identify seven key actions for the upcoming years that aim at (1) improving our attractiveness as an employer; (2) improving diversity and inclusion; (3) improving the execution of the PhD program; (4) expanding our staff capacity; (5) aligning our office space with the optimal way of working; (6) articulating the scientific profile; and (7) boosting our scientific and societal impact.

Academic Leadership, Module 1

Last week I participated in the first (two day) module of a six month TU Delft course on “Academic Leadership” — a course so successful it has been taught every single year for the past 32 years.

Maybe the most impressive content comes from the participants themselves (16 in total this year), who serve in different leadership roles at TU Delft. Participants can bring in “cases” they are currently struggling with — my case relates to moving my department to a new building (with less space for the 150 people involved). The participants can ask questions about these cases, often reflecting their own experience in dealing with similar cases. The questions not only help to drill down to the essence of the case, but also to the (possibly deeply personal) reasons behind the struggle at hand.

The format for this participatory content is that of “intervision“, which in English translates to “peer supervision”. It is a technique common among (mental) health care professionals, to exchange their experiences, to analyze how they handle a given complex situation, and to reflect collectively on their professional conduct.

The intervision in this course takes place in smaller groups of four, under the guidance of a coach. In a series of sessions, each participant gets one afternoon to present his or her case, and to discuss it in depth in a trusted, fully confidential setting. A few years back I participated in such an intervision, and I look forward to doing this again.

The actual course content of the first module came from Mathieu Weggeman, a professor and consultant who specializes in management of knowledge-intensive organizations. His lecture carried the title of his book Managing Professionals? Don’t!, emphasizing that professionals usually work best when their managers take a step back. No more “planning and control”, but a focus on shared ambition and employee expertise. I’m sure this resonates with many academics.

Weggeman spent considerable time discussing the characteristics of leaders in excellent professional organizations. Such leaders:

  • develop, together with all employees, a shared ambition;
  • inspire people, and involve them in the organization’s strategy to materialize the ambition;
  • communicate fairly and timely: they are available, and they listen (think management by walking around);
  • are clear about the desired output, and offer clear feedback;
  • are assertive towards employees who are not good at their job anymore;
  • function as “heat shield” against “noise from above”
  • have an authoritative yet serving and humble attitude

Weggeman connected this to a quote from Laozi (老子, 6th century BC):

A leader is best when people barely know he exists,
not so good when people obey and acclaim him,
worse when they despise him.
But a good leader, who talks little, when the work is done, his aim fulfilled,
they will say:
We did it ourselves.

Weggeman also discussed tools to diagnose and design organizations. Such tools need to distinguish (1) setting goals, (2) designing the organization, and (3) executing a strategy to meet the stated objectives — in Dutch nicely summarized as richten, inrichten, verrichten. Weggeman explained how such activities can be influenced through organizational “design variables”, which he (loosely) based on McKinsey’s 7S Framework. This framework distinguishes seven elements, described as (wikipedia):

  • Strategy: Purpose of the business and the way the organization seeks to enhance its competitive advantage.
  • Structure: Division of activities; integration and coordination mechanisms.
  • Systems: Formal procedures for measurement, reward and resource allocation.
  • Shared Values: Included in culture by Weggeman, who also includes in culture the way of working derived from these values.
  • Skills: The organization’s core competencies and distinctive capabilities.
  • Staff: Organization’s human resources, demographic, educational and attitudinal characteristics.
  • Style: Typical behavior patterns of key groups, such as managers, and other professionals

McKinseys 7S Framework

The basic premise of this framework is that these seven internal aspects of an organization need to be aligned, and that they are interrelated: Changing one element will affect the others.

As any good management consultant, Weggeman was full of quotes. To explain the need for a shared ambition, he quoted Nietzche:

He who has a ‘why’ to live for can bear almost any ‘how’.
(“Hat man sein warum des Lebens, so verträgt man sich fast mit jedem wie”, translation by Frankl)

As an academic, it is easy to get lost in the fights of the “how” (getting tenure, submitting a paper, writing a review, applying for funding, managing the class room, handling Blackboard Brightspace, etc., etc.). And naturally, it is our collective duty to improve the ‘how’ wherever we can.

But our ‘why’ is clear: Driven by curiosity, we train young people to become the world’s leading computer scientists and software engineers, and we push the boundaries of what the world knows about computer science. And this we want, in the words of the late David Notkin, “so that society can benefit even more from the amazing potential of software.”