Factors Affecting Software Developer's Performance in the 90s.
A Literature Review
As a Researcher, you have to read—a lot. Your first year of your Ph.D. is probably spent reading about how to read and write and reading material relevant to your Research and thesis. Of course, you also tend to procrastinate over your thesis, but that's for another time. (or that may be just me!)
As I went down the rabbit hole trying to find academic literature on using Behavioral Science or Behavioral Concepts in Software Engineering, I came across a paper called, Factors Affecting Software Developers Performance: An Integrated Approach from the University of Florida. The article was published in 1992 in MIS Quarterly, and because I find it highly relevant today, I decided to do a very long version of the literature review.
A high-level overview of the paper
The paper builds on prior Research on Software Developers' performance and extends it by integrating theoretical behavioral approaches like expectancy theory, goal-setting theory, and individual characteristics.
The findings indicate that goal-setting theory has complex implications for software development performance. In contrast, individual ability has the most substantial direct effect on perceived performance, twice as much as the actual effort and characteristics of the task (or, in product terms, requirements).
What's extremely interesting is that the data to back up this study was gathered from 3 different organisations across three different regions of the US, with a diverse spectrum of software projects ranging from the Public Sector to Telecoms to Finance.
The Theory
Bear with me while we define the three behavioral concepts mentioned above. These are relevant because they have often been used to analyse the relationship between different elements that affect individual motivation and performance.
Expectancy Theory (Baker et al., 1989, Brownell and McInnes, 1986)
Expectancy Theory is commonly used to analyze motivational issues. The Research supporting this theory is based on the idea that highly motivated individuals tend to perform at higher levels, particularly when positive outcomes are associated with their behavior.
Bringing this back to Software Engineering, we can argue that highly motivated and satisfied engineers will perform better than those that aren't.
This is where tools like Developers Surveys and Teams Health Checks can go a long way to assess the level of motivation and satisfaction.
Goal-Setting Theory (Locke and Latham, 1990)
According to Research on goal-setting theory, the expected success level may be related to how precise, and complex work requirements are. If the tasks are unclear, it creates ambiguity, which leads to task anxiety, hesitation in making decisions, and lower performance.
Taking this out of the academic world and into Engineering, when the outcome or goal isn't clear enough, it can negatively impact Productivity.
This is particularly relevant in the light of a new Survey released by Gartner, where they interviewed Engineering Leaders on assessing Developer Productivity. 52% of respondents said they measure it by the business outcomes achieved.(!)
Individual Characteristics (Mitchell, 1974; 1979; 1982)
The third and final concept relates to individual characteristics. For this study, these were defined as a need for achievement (the extent to which we value success), locus of control (referring to who we perceive is in control: external factors or ourselves), and high self-esteem (a person's sense of self-worth).
The authors argue that an individual's need for achievement and self-esteem can balance the relationship between productivity and goals.
Frankly, this one was too theoretical for me because while I can see how drive and high self-esteem can improve Productivity; these are equally harder to measure and define in a work environment.
The Research
For this study, the authors used data from 3 different organisations of different sizes and industries. The Developers that participated in this Research were involved in the Development process from Discovery (yes, they did Product Discovery in 1992 as well) to Release.
They used questionnaires or what we call today Developer Surveys, and the data was obtained in person for two organisation and via email for one organisation; in total, 335 responses. ( 27% female and 73% male).
Below I have included a sample from the questionnaire:
Source: Rasch, R. H., & Tosi, H. L. (1992). Factors Affecting Software Developers’ Performance: An Integrated Approach. MIS Quarterly, 16(3), 395–413. https://doi.org/10.2307/249535
Summary of Findings and Conclusion:
Individual ability had the highest direct positive effect on performance
The need for high achievement came second as positively related to effort and perceived performance.
And finally, goal clarity had the lowest direct positive impact on Developer's performance.
In short, this study shows that performance is directly related to skills, a high need for achievement, and clarity of outcomes.
Limitations and gaps in the Research:
335 engineers is not a big pool, making it challenging to generalise the findings to a larger pool of Developers.
Additionally, we do not know the size and culture set up for the organizations chosen, which can ultimately influence performance.
Finally, the Researchers used a structural equation model as a modelling technique. Still, because the data is so cross-sectional, causality between variables should be perceived cautiously. (as indicated by authors too).
My take on it:
This is a fascinating study, even though it's > 30 years old because it uses behavioral theories to analyse variables in performance for a Computer Science role. It also highlights the importance of taking a holistic approach to measuring performance and considering both social science and quantitative variables.
Because the experiments were generalised across different organizations and sectors, the findings are also highly relevant today.
As a Product Manager working on Developer Tools, Experience, or Enablement (whatever the term is in your organisation), we should always want to be interested in Engineering Health Checks and constantly run Satisfaction Surveys. This can be a way to assess a level of motivation.
Recently I learned about Self-Reported Productivity surveys and I am considering introducing this, as I have yet to experiment with those. I will report back with the results.
Writing the Surveys, it's also a different story, which we will tackle in the future with practical examples
The best thing we can do is have a continuous feedback loop in Engineering to try and understand what works and what doesn't in the current context of the organisation.
See you next Friday!
Some exciting resources, related to this:
Getting more from your Team Health Checks, from Spotify Engineering
An interesting preliminary use of GitHub Copilot, from GitHub