The Collaborative Software Architecture Course

After four exciting years of Teaching Software Architecture Using GitHub, we decided to write a paper reflecting on the course and our experiences, and submit it to SIGCSE, the flagship conference of the ACM Special Interest Group on Computer Science Education, typically attended by more than a thousand educators from around the world.

We’re very happy that our paper was immediately accepted!

In the paper, we identify three challenges in teaching software architecture:

  • C1: The theory of software architecture (design principles, tradeoffs, architectural patterns, product lines, etc) is often very abstract and therefore hard for a student to master.
  • C2: The problems of software architecture are only visible at scale, and disappear once small example systems are used.
  • C3: A software architect needs a combination of technical and social skills: software architecture is about communication between stakeholders, and the architect needs to be able to achieve and explain consensus.

To address these challenges, the paper proposes a collaborative approach to teaching software architecture. In particular, we report how we organized our software architecture course according to the following principles:

  • Embrace open source: Students pick an open source system of choice and study its architecture. Students use it to learn how to apply architectural theories to realistic systems (C1, C2).
  • Embrace collaboration: Students work in teams of four to study one system in depth (C3).
  • Embrace open learning: Teams share all of their work with other students. Furthermore, students share their main result with the open source community: their architectural description is published as a chapter in an online book resulting from the course (C3).
  • Interact with the architects: Students are required to offer contributions (in the form of GitHub pull requests) to the open source projects, which will expose them to feedback from actual integrators and architects of the open source projects (C1, C2, C3).
  • Combine breadth and depth: Students dive deeply in the system they analyze themselves, and learn broadly from the analyses conducted and presented by other teams (C1, C3).

DESOSA 2016 book cover

In 2016 the resulting book (created in markdown and git using gitbook) described the architectures of 21 open source systems, including Ember.js, Karma, Neo4j, and SonicPi. The chapters are based both on existing architectural theories (such as architectural views, product lines, and technical debt), as well as the students’ first hand experiences in making actual contributions (merged pull requests) to the open source systems under study.

SIGCSE Abstract

Teaching software architecture is hard. The topic is abstract and is best understood by experiencing it, which requires proper scale to fully grasp its complexity. Furthermore, students need to practice both technical and social skills to become good software architects. To overcome these teaching challenges, we developed the Collaborative Software Architecture Course. In this course, participants work together to study and document a large, open source software system of their own choice. In the process, all communication is transparent in order to foster an open learning environment, and the end-result is published as an online book to benefit the larger open source community.

We have taught this course during the past four years to classes of 50-100 students each. Our experience suggests that: (1) open source systems can be successfully used to let students gain experience with key software architecture concepts, (2) students are capable of making code contributions to the open source projects, (3) integrators (architects) from open source systems are willing to interact with students about their contributions, (4) working together on a joint book helps teams to look beyond their own work, and study the architectural descriptions produced by the other teams.

Arie van Deursen, Maurício Aniche, Joop Aué, Rogier Slag, Michael de Jong, Alex Nederlof and Eric Bouwers. “A Collaborative Approach to Teaching Software Architecture.” Proceedings of the 48th ACM Technical Symposium on Computer Science Education (SIGCSE), March 2017, Seattle, USA.

You can download the paper from the TU Delft institutional repository, or have a look at the slides we used at our SIGCSE 2017 presentation.

Embedded Software Development with C Language Extensions

Arie van Deursen, with Markus Voelter, Bernd Kolb, and Stephan Eberle.

In embedded systems development, C remains the dominant programming language, because it permits writing low level algorithms and producing efficient binaries. Unfortunately, the price to pay for this is limited support for explicit and safe abstractions.

To overcome this, engineers at itemis and fortiss created mbeddr: an extensible version of C that comes with extensions relevant to embedded software development. Examples include explicit support for state machines, variability management, physical units, interfaces and components, or unit testing. The extensions are supported by an IDE created through JetBrains MPS. Furthermore, mbeddr users can introduce their own extensions.

To me, the ideas under mbeddr are extremely appealing. But I also had concerns: Would this work in practice? Does this scale to real world embedded systems? What are the benefits of such an approach? What are the problems?

Therefore, when Markus Voelter, lead architect of mbeddr invited me to join in a critical evaluation of a system created with mbeddr that they just shipped, I happily accepted. Eventually, this resulted in our paper Using C Language Extensions for Developing Embedded Software: A Case Study, which was accepted for publication and presentation at OOPSLA 2015.

The subject system built with mbeddr is an electricity smart meter, which continuously senses the instantaneous voltage and current on a mains line using analog front ends and analog-to-digital converters. It’s mbeddr implementation consists of 80 interfaces and 167 components, corresponding to roughly 44,000 lines of C code.

Main layers, sub-systems, and components of the smart metering system.

Main layers, sub-systems, and components of the smart metering system.

Our goal in analyzing this system was to find out the degree to which C language extensions (as implemented in mbeddr) are useful for developing embedded software. We adopted the case study research method to investigate the use of mbeddr in an actual commercial project, since the true risks and benefits of language extensions can be observed only in such projects. Focussing on a single case allows us to provide significant details about that case.

To achieve this goal, we investigated the following aspects of the smart metering system:

  1. Complexity: Are the abstractions provided by mbeddr beneficial for mastering the complexity encountered in a real-world embedded system? Which additional abstractions would be needed or useful?
  2. Testing: Can the mbeddr extensions help with testing the system? In particular, is hardware-independent testing possible to support automated, continuous integration and build? Is incremental integration and commissioning supported?
  3. Overhead: Is the low-level C code generated from the mbeddr extensions efficient enough for it to be deployable onto a real-world embedded device?
  4. Effort: How much effort is required for developing embedded software with mbeddr?

The detailed analysis and answers are in the paper. Our main findings are the following:

  • The extensions help mastering complexity and lead to software that is more testable, easier to integrate and commission, and that is more evolvable.
  • Despite the abstractions introduced by mbeddr, the additional overhead is very low and acceptable in practice.
  • The development effort is reduced, particularly regarding evolution and commissioning.

In our paper, we also devote four pages to potential threats to the validity of our findings. Most importantly, in our experience with this case study and other projects, introducing mbeddr into an organization may be difficult, despite these benefits, due to a lack of developer skills and the need to adapt the development process.

The key insight for me is that mbeddr can help bring down one of the biggest cost and risk factors in embedded systems development, which is the integration and commissioning on the target hardware. Typically, this phase accounts for 40-50% of the project cost; for the smart meter system this was 13%. This was achieved by extensive unit and integration testing, using interfaces that could be instantiated both in a test as well as a target hardware environment.

Continuous integration is not just about the use of a continuous integration server. It is primarily about carefully modularizing the system into components that can be tested independently in different environments. Unfortunately, modularization is hard, especially in languages without explicit modularization primitives. Our study shows how extending C with language constructs can help to devise a modular, testable architecture, substantially reducing integration and commissioning costs.

For more information, see:

  • Markus Völter, Arie van Deursen, Bernd Kolb, Stephan Eberle. Using C Language Extensions for Developing Embedded Software: A Case Study. OOPSLA/SPLASH 2015 (pdf).
  • Presentation at OOSPLA 2015 by Markus Voelter (youtube, slides)
  • Information on this paper at the OOPSLA program pages.

Delft Students on Software Architecture: DESOSA 2015

With Rogier Slag.

This year, we taught another edition of the TU Delft Teaching Software Architecture — With GitHub course.

We are proud to announce the resulting on line book: Delft Students on Software Architecture is a collection of architectural descriptions of open source software systems written by students from Delft University of Technology during a master-level course taking place in the spring of 2015.

desosa 2015 book cover

At the start of the course, teams of 3-4 students could adopt a project of choice on GitHub. The projects selected had to be sufficiently complex and actively maintained (one or more pull requests merged per day).

During a 10 week period, the students spent one third of their time on this course,and engaged with these systems in order to understand and describe their software architecture.

Inspired by Brown and Wilsons’ Architecture of Open Source Applications, we decided to organize each description as a chapter, resulting in the present online book.

Recurring Themes

The chapters share several common themes, which are based on smaller assignments the students conducted as part of the course. These themes cover different architectural ‘theories’ as available on the web or in textbooks. The course used Rozanski and Woods’ Software Systems Architecture, and therefore several of their architectural viewpoints and perspectives recur.

The first theme is outward looking, focusing on the use of the system. Thus, many of the chapters contain an explicit stakeholder analysis, as well as a description of the context in which the systems operate. These were based on available online documentation, as well as on an analysis of open and recently closed issues for these systems.

A second theme involves the development viewpoint, covering modules, layers, components, and their inter-dependencies. Furthermore, it addresses integration and testing processes used for the system under analysis.

A third recurring theme is variability management. Many of today’s software systems are highly configurable. In such systems, different features can be enabled or disabled, at compile time or at run time. Using techniques from the field of product line engineering, several of the chapters provide feature-based variability models of the systems under study.

A fourth theme is metrics-based evaluation of software architectures. Using such metrics architects can discuss (desired) quality attributes (performance, scaleability, maintainability, …) of a system quantitatively. Therefore various chapters discuss metrics and in some cases actual measurements tailored towards the systems under analysis.

First-Hand Experience

Last but not least, the chapters are also based on the student’s experience in actually contributing to the systems described. As part of the course over 75 pull requests to the projects under study were made, including refactorings (Jekyll 3545, Docker 11350, Docker 11323, Syncany 391), bug fixes
(Diaspora 5714, OpenRA 7486, OpenRA 7544, Kodi 6570), and helpful documentation such as a Play Framework screen cast.

Through these contributions the students often interacted with lead developers and architects of the systems under study, gaining first-hand experience with the architectural trade-offs made in these systems.

Enjoy!

Working with the open source systems and describing their architectures has been a great experience, both for the teachers and the students.

We hope you will enjoy reading the DESOSA chapters as much as we enjoyed writing them.

Teaching Software Architecture: with GitHub!

Arie van Deursen, Alex Nederlof, and Eric Bouwers.

When teaching software architecture it is hard to strike the right balance between practice (learning how to work with real systems and painful trade offs) and theory (general solutions that any architect needs to thoroughly understand).

To address this, we decided to try something radically different at Delft University of Technology:

GitHub Octocat

  • To make the course as realistic as possible, we based the entire course on GitHub. Thus, teams of 3-4 students had to adopt an active open source GitHub project and contribute. And, in the style of “How GitHub uses GitHub to Build GitHub“, all communication within the course, among team members, and with external stakeholders took place through GitHub where possible.

Rozanski & Woods

  • Furthermore, to ensure that students actually digested architectural theory, we made them apply the theory to the actual project they picked. As sources we used literature available on the web, as well as the software architecture textbook written by Nick Rozanski and Eoin Woods.

To see if this worked out, here’s what we did (the course contents), how we did it (the course structure, including our use of GitHub), and what most of us liked, or did not like so much (our evaluation).

Course Contents

GitHub Project Selection

At the start of the course, we let teams of 3-4 students adopt a project of choice on GitHub. We placed a few constraints, such as that the project should be sufficiently complex, and that it should provide software that is actually used by others — there should be something at stake.

Based on our own analysis of GitHub, we also provided a preselection of around 50 projects, which contained at least 200 pull requests (active involvement from a community) and an automated test suite (state of the art development practices).

The projects selected by the students included Netty, Jenkins, Neo4j, Diaspora, HornetQ, and jQuery Mobile, to name a few.

Making Contributions

To get the students involved in their projects, we encouraged them to make actual contributions. Several teams were successful in this, and offered pull requests that were actually merged. Examples include an improvement of the Aurelius Titan build process, or a fix for an open issue in HornetQ. Other contributions were in the form of documentation, for example one team created a (popular) screencast on how to build a chat client/server system with Netty in under 15 minutes, posted on Youtube.

The open source projects generally welcomed these contributions, as illustrated by the HornetQ reaction:

Thanks for the contribution… I can find you more if you need to.. thanks a lot! :+1:

And concerning the Netty tutorial:

Great screencast, fantastic project.

Students do not usually get this type of feedback, and were proud to have made an impact. Or, as the CakePHP team put it:

All in all we are very happy with the reactions we got, and we really feel our contribution mattered and was appreciated by the CakePHP community.

DelftSWA twitter account

To make it easier to establish contact with such projects, we setup a decidcated @delftswa Twitter account. Many of the projects our students analyzed have their own Twitter accounts, and just mentioning a handle like @jenkinsci or @cakephp helped receive the project’s attention and establish the first contact.

Identifying Stakeholders

One of the first assignments was to conduct a stakeholder analysis:
understanding who had an interest in the project, what their interest was, and which possibly conflicting needs existed.

To do this, the students followed the approach to identify and engage stakeholders from Rozanski and Woods. They distinguish various stakeholder classes, and recommend looking for stakeholders who are most affected by architectural decisions, such as those who have to use the system, operate or manage it, develop or test it, or pay for it.

Stakeholder analysis for Aurelius Titan

Aurelius Stakeholders (click to enlarge)

To find the stakeholders and their architectural concerns, the student teams analyzed any information they could find on the web about their project. Besides documentation and mailing lists, this also included an analysis of recent issues and pull requests as posted on GitHub, in order to see what the most pressing concerns of the project at the moment are and which stakeholders played a role in these discussions.

Architectural Views

Since Kruchten’s classical “4+1 views” paper, it has been commonly accepted that there is no such thing as the architecture. Instead, a single system will have multiple architectural views.

Following the terminology from Rozanski and Woods, stakeholders have different concerns, and these concerns are addressed through views.
Therefore, with the stakeholder analysis in place, we instructed students to create relevant views, using the viewpoint collection from Rozanski and Woods as starting point.

Viewpoints from Rozanski & Woods

Rozanski & Woods Viewpoints

Some views the students worked on include:

  • A context view, describing the relationships, dependencies, and interactions between the system and its environment. For example, for Diaspora, the students highlighted the relations with users, `podmins’, other social networks, and the user database.

Diaspora Context Diagram

  • A development view, describing the system for those involved in building, testing, maintaining, and enhancing the system. For example, for neo4j, students explained the packaging structure, the build process, test suite design, and so on.

Neo4J Module View

  • A deployment view, describing the environment into which the system will be deployed, including the dependencies the system has on its runtime environment. For GitLab, for example, the students described the installations on the clients as well as on the server.

GitLab Deployment View

Besides these views, students could cover alternative ones taken from other software architecture literature, or they could provide an analysis of the perspectives from Rozanski and Woods, to cover such aspects as security, performance, or maintainability.

Again, students were encouraged not just to produce architectural documentation, but to challenge the usefulness of their ideas by sharing them with the projects they analyzed. This also forced them to focus on what was needed by these projects.

For example, the jQuery Mobile was working on a new version aimed at realizing substantial performance improvements. Therefore, our students conducted a series of performance measurements comparing two versions as part of their assignment. Likewise, for CakePHP, our students found the event system of most interest, and contributed a well-received description of the new CakePHP Events System.

Software Metrics

For an architect, metrics are an important tool to discuss (desired) quality attributes (performance, scaleability, maintainability, …) of a system quantitatevely. To equip students with the necessary skills to use metrics adequately, we included two lectures on software metrics (based on our ICSE 2013 tutorial).

Students were asked to think of what measurements could be relevant for their projects. To that end, they first of all had to think of an architecturally relevant goal, in line with the Goal/Question/Metric paradigm. Goals picked by the students included analyzing maintainability, velocity in responding to bug reports, and improving application responsiveness.

Subsequently, students had to turn their project goal into 3 questions, and identify 2 metrics per question.

Brackets Analyzability

Brackets Analyzability

Many teams not just came up with metrics, but also used some scripting to actually conduct measurements. For example, the Brackets team decided to build RequireJS-Analyzer, a tool that crawls JavasScript code connected using RequireJS and analyzes the code, to report on metrics and quality.

Architectural Sketches

Being an architect is not just about views and metrics — it is also about communication. Through some luck, this year we were able to include an important yet often overlooked take on communication: The use of design sketches in software development teams.

We were able to attract Andre van der Hoek (UC Irvine) as guest speaker. To see his work on design sketches in action, have a look at the Calico tool from his group, in which software engineers can easily share design ideas they write at a sketch board.

In one of the first lectures, Andre presented Calico to the students. One of the early assignments was to draw any free format sketches that helped to understand the system under study.

The results were very creative. The students freed themselves from the obligation to draw UML-like diagrams, and used colored pencils and papers to draw what ever they felt like.

HornetQ Design Sketch

For example, the above picture illustrates the HornetQ API; Below you see the Jenkins work flow, including dollars floating from customers the contributing developers.

Jenkins Design Sketch

Lectures From Industry

Last but not least, we wanted students to meet and learn from real software architects. Thus, we invited presentations from the trenches — architects working in industry, sharing their experience.

For example, in 2013 we had TU Delft alumnus Maikel Lobbezoo, who presented the architectural challenges involved in building the payment platform offered by Adyen. The Adyen platform is used to process gazilions of payments per day, acrross the world. Adyen has been one of the fastest growing tech companies in Europe, and its success can be directly attributed to its software architecture.

Adyen Flow

Maikel explained the importance of having a stateless architecture, that is linearly scalable, redundant and based on push notifications and idempotency. His key message was that the successful architect possesses a (rare) combination of development skills, communicative skills, and strategical bussiness insight.

Our other guest lecture covered cyber-physical systems: addressing the key role software plays in safety-critical infrastructure systems such as road tunnels — a lecture provided by Eric Burgers from Soltegro. Here the core message revolved around traceability to regulations, the use of model-based techniques such as SysML, and the need to communicate with stakeholders with little software engineering experience.

Course Structure

Constraints

Some practical constraints of the course included:

  • The course was worth 5 credit points (ECTS), corresponding to 5 * 27 = 135 hours of work per student.
  • The course run for half a semester, which is 10 consecutive weeks
  • Each week, there was time for two lectures of 90 minutes each
  • We had a total of over 50 students, eventually resulting in 14 groups of 3-4 students.
  • The course was intended four 4th year students (median age 22) who were typically following the TU Delft master in Computer Science, Information Architecture, or Embedded Systems.

GitHub-in-the-Course

All communication within the course took place through GitHub, in the spirit of the inspirational video How GitHub uses GitHub to Build GitHub. As a side effect, this eliminated the need to use the (not so popular) Blackboard system for intra-course communication.

From GitHub, we obtained a (free) delftswa organization. We used it to host various repositories, for example for the actual assignments, for the work delivered by each of the teams, as well as for the reading material.

Each student team obtained one repository for themselves, which they could use to collaboratively work on their assignments. Since each group worked on a different system, we decided to make these repositories visible to the full group: In this way, students could see the work of all groups, and learn from their peers how to tackle a stakeholder analysis or how to assess security issues in a concrete system.

Having the assignment itself on a GitHub repository not only helped us to iteratively release and improve the assignment — it also gave students the possibility to propose changes to the assignment, simply by posting a pull request. Students actually did this, for example to fix typos or broken links, to pose questions, to provide answers, or to start a discussion to change the proposed submission date.

We considered whether we could make all student results public. However, we first of all wanted to provide students with a safe environment, in which failure could be part of the learning process. Hence it was up to the students to decide which parts of their work (if any) they wanted to share with the rest of the world (through blogs, tweets, or open GitHub repositories).

Time Keeping

One of our guiding principles in this course was that an architect is always eager to learn more. Thus, we expected all students to spend the full 135 hours that the course was supposed to take, whatever their starting point in terms of knowledge and experience.

This implies that it is not just the end result that primarily counts, but the progress the students have made (although there is of course, a “minimum” result that should be met at least).

To obtain insight in the learning process, we asked students to keep track of the hours they spent, and to maintain a journal of what they did each week.

For reasons of accountability such time keeping is a good habit for an architect to adopt. This time keeping was not popular with students, however.

Student Presentations

We spent only part of the lecturing time doing class room teaching.
Since communication skills are essential for the successful architect,
we explicitly allocated time for presentations by the students:

  • Half-way all teams presented a 3-minute pitch about their plans and challenges.
  • At the end of the course we organized a series of 15 minute presentations in which students presented their end results.

Each presentation was followed by a Q&A round, with questions coming from the students as well as an expert panel. The feedback concerned the architectural decisions themselves, as well as the presentation style.

Grading

Grading was based per team, and based on the following items:

  • Final report: the main deliverable of each team, providing the relevant architectural documentation created by the team.
  • Series of intermediate (‘weekly’) reports corresponding to dedicated assignments on, e.g, metrics, particular views, or design sketches. These assignments took place in the first half of the course, and formed input for the final report;
  • Team presentations

Furthermore, there was one individual assignment, in which each student had to write a review report for the work of one of the other teams. Students did a remarkably good (being critical as well as constructive) job at this. Besides allowing for individual grading, this also ensured each team received valuable feedback from 3-4 fellow students.

Evaluation

All in all, teaching this course was a wonderful experience. We gave the students considerable freedom, which they used to come up with remarkable results.

A key success factor was peer learning. Groups could learn from each other, and gave feedback to each other. Thus, students not just made an in depth study of the single system they were working on, but also learned about 13 different architectures as presented by the other groups. Likewise, they not just learned about the architectural views and perspectives they needed themselves, but learned from their co-students how they used different views in different ways.

The use of GitHub clearly contributed to this peer learning. Any group could see, study, and learn from the work of any other group. Furthermore, anyone, teachers and students alike, could post issues, give feedback, and propose changes through GitHub’s facilities.

On the negative side, while most students were familiar with git, not all were. The course could be followed using just the bare minimum of git knowledge (add, commit, checkout, push, pull), yet the underlying git magic sometimes resulted in frustration with the students if pull requests could not be merged due to conflicts.

An aspect the students liked least was the time keeping. We are searching for alternative ways of ensuring that time is spent early on in the project, and ways in which we can assess the knowledge gain instead of the mere result.

One of the parameters of a course like this is the actual theory that is discussed. This year, we included views, metrics, sketches, as well as software product lines and service oriented architectures. Topics that were less explicit this year were architectural patterns, or specific perspectives such as security or concurrency. Some of the students indicated that they liked the mix, but that they would have preferred a little more theory.

Conclusion

In this course, we aimed at getting our students ready to take up a leading role in software development projects. To that end, the course put a strong focus on:

  • Close involvement with open source projects where something was at stake, i.e., which were actually used and under active development;
  • The use of peer learning, leveraging GitHub’s social coding facilities for delivering and discussing results
  • A sound theoretical basis, acknowleding that architectural concepts can only be grasped and deeply understood when trying to put them into practice.

Next year, we will certainly follow a similar approach, naturally with some modifications. If you have suggestions for a course like this, or are aware of similar courses, please let us know!

Acknowledgments

A big thank you to all students who participated in IN4315 in 2013,
to jury members Nicolas Dintzner, Felienne Hermans, and Georgios Gousios, and guest lecturers Andre van der Hoek, Eric Burgers, Daniele Romano, and Maikel Lobbezoo for shaping this course!

Further Reading

  1. Zach Holman. How GitHub uses GitHub to Build GitHub. RubyConf, September 2011.
  2. Nick Rozanski and Eoin Woods. Software Systems Architecture: Working with Stakeholders Using Viewpoints and Perspectives. Addison-Wesley, 2012, 2nd edition.
  3. Diomides Spinellis and Georgios Gousios (editors). Beautiful Architecture: Leading Software Engineers Explain How They Think. O’Reilly Media, 2009.
  4. Amy Brown and Greg Wilson (editors). The Architecture of Open Source Applications. Volumes 1-2, 2012.
  5. Software Architecture Wikipedia page (thoughtfully and substantially edited by the IFIP Working Group on Software Architecture in 2012)
  6. Eric Bouwers. Metric-based Evaluation of Implemented Software Architectures. PhD Thesis, Delft University of Technology, 2013.
  7. Remco de Boer, Rik Farenhorst, Hans van Vliet: A Community of Learners Approach to Software Architecture Education. CSEE&T, 2009.
  8. Earlier use of GitHub in the class room by Neil Ernst while at UBC: Using GitHub for 3rd Year Software Engineering, and Teaching Advanced Software Engineering.

© Text: Arie van Deursen, Alex Nederlof, and Eric Bouwers, 2013.

© Architectural views: students of the TU Delft IN4315 course, 2013.

David Notkin on Why We Publish

This week David Notkin (1955-2013) passed away, after a long battle against cancer. He was one of my heroes. He did great research on discovering invariants, reflexion models, software architecture, clone analysis, and more. His 1986 Gandalf paper was one of the first I studied when starting as a PhD student in 1990.

December 2011 David sent me an email in which he expressed interest to do a sabbatical in our TU Delft Software Engineering Research Group in 2013-2014. I was as proud as one can be. Unfortunately, half a year later he had to cancel his plans due to his health.

David was loved by many, as he had a genuine interest in people: developers, software users, researchers, you. And he was a great (friendly and persistent) organizer — 3 weeks ago he still answered my email on ICSE 2013 organizational matters.

In February 2013, he wrote a beautiful editorial for the ACM Transactions on Software Engineering and Methodology, entitled Looking Back. His opening reads: “It is bittersweet to pen my final editorial”. Then David continues to address the question why it is that we publish:

“… I’d like very much for each and every reader, contributor, reviewer, and editor to remember that the publications aren’t primarily for promotions, or for citation counts, or such.

Rather, the intent is to make the engineering of software more effective so that society can benefit even more from the amazing potential of software.

It is sometimes hard to see this on a day-to-day basis given the many external pressures that we all face. But if we never see this, what we do has little value to society. If we care about influence, as I hope we do, then adding value to society is the real measure we should pursue.

Of course, this isn’t easy to quantify (as are many important things in life, such as romance), and it’s rarely something a single individual can achieve even in a lifetime. But BHAGs (Big Hairy Audacious Goals) are themselves of value, and we should never let them fade far from our minds and actions.”

Dear David, we will miss you very much.


See also:


Speaking in Irvine on Metrics and Architecture

End of last year I was honored to receive an invitation to present in the Distinguished Speaker Series at the Insitute for Software Research at University of California at Irvine.

I quickly decided that the topic to discuss would be our research on software architecture, and in particular our work on metrics for maintainability.

Irvine is one of the world’s leading centers on research in software architecture. The Institute of Software Research is headed by Richard Taylor, who supervised Roy Fielding when he wrote his PhD thesis covering the REST architectural style, and Nenad Medvidovic during his work on architectural description laguages. Current topics investigated at Irvine include design and collaboration (André van der Hoek, and David Redmiles of ArgoUML fame), software analyis and testing (James Jones), and programming laguages (Cristina Lopes), to name a few. An overview of the group’s vision on software architecture can be found in their recently published textbook. In short, I figured that if there is one place to present our software architecture research it must be Irvine.

The talk (90 minutes) itself will be loosely based on my keynote at the Brazilian Software Engineering Symposium (SBES 2012), which in turn is based on joint research with Eric Bouwers and Joost Visser (both from SIG). I’ll post the slides when I’m done. The full slides are available on speakerdeck, but here’s the storyline along with some references.

A Software Risk Assessment (from )

A Software Risk Assessment (source: ICSM 2009)

The context of this research is a software risk assessment, in which a client using a particular system seeks independent advice (from a consultant) on the technical quality of the system as created by an external supplier.

How can the client be sure that the system made for him is of good quality? In particular, will it be sufficiently maintainable, if the business context of the system in question changes? Will it be easy to adapt the system to the ever changing world?

In situations like these, it is quintessential to be able to make objective, evidence-based statements about the maintainability of the system in question.

Is this possible? What role can metrics play? What are their inherent limitations? How can we know that a metric indeed captures certain aspects of maintainability? How should metric values be interpreted? How should proposals for new metrics be evaluated?

Simple answers to these questions do not exist. In this talk, I will summarize our current progress in answering these questions.

I will start out by summarizing four common pitfalls when using metrics in a software development project. Then, I will describe a metrics framework in which metrics are put into context by means of benchmarking and a quality model. Subsequently, I’ll zoom in on architectural metrics, focusing on metrics for encapsulation. I will discuss a proposal for a new metric, as well as its evaluation. The evaluation comprises both a quantitative assessment (using repository-mining) of its construct validity (doest it measure encapsulation?), as well as qualitative assessments of the usefulness in practice (by interviewing consultants who applied the metrics in their day to day work).

Based on this, I will reflect on the road ahead for empirical research in software metrics and architecture, emphasizing the need for shared datasets, as well as the use of qualitative research methods to evaluate practical impact.

The talk is scheduled for Friday March 15, in Irvine — I sincerely hope to see you there!

If you can’t make it, Eric Bouwers and I will present a 3.5-hour tutorial based on this same material at ICSE 2013, in May in San Francisco. The tutorial will be more interactive, taking your experience into account as well where possible, and it will have a stronger emphasis on metrics (based on SIG’s 10 year experience with using metrics in industry). Register now for our tutorial, and looking forward to seeing you there!


See also: