Software developers are the engine of growth and innovation that powers the products on which everyone relies. Developers are a company’s most valuable resource. The demand for software engineers worldwide will rise by 22% between 2020 and 2030, even with recent cutbacks. Yet there remains a tremendous squandering of this precious resource.
Wasted developer time and effort, which includes inefficient processes and code rework, is often referred to as “developer toil.” According to Stripe, developers spend up to half their work week in toil, with some estimates as high as 80%. Toil costs businesses over $85 billion per year in wasted developer effort.
Companies are struggling to catch on to the problem in their engineering teams, and many have yet to act on it, but the impact of excessive developer toil is evident in staff retention rates. Toil is incredibly frustrating to developers. Over 70% of developers report that cycles of rework make them unhappy and result in seeking other employment opportunities. Professional frustration contributes to an unprecedented average turnover rate among developers of 57.3%, according to the 2021 Bureau of Labor Statistics report. And while there have been recent layoffs in technology companies, the demand for developers remains high.
In this article, we will focus on developer experience (DX) and reducing developer toil associated with software complexity and defect remediation. Avoidable rework is 40% of the developer’s work week, constituting most of the toil. Developers spend a lot of time attempting to understand runtime behavior and implement coding fixes for defects in high-priority areas, such as performance and security. These are customer-impacting issues that businesses and developers alike wish to avoid.
The causes of developer toil
A major contributing factor to developer toil is the growing size and complexity of modern software applications. Application sizes grew in the last decade to an average of 300,000 lines of code. Today’s software applications typically comprise vast code bases and thousands of connected services and microservices, immense surface areas of functionality that could be impacted with each code change. These systems are too complex to predict how code changes will affect system behavior intuitively, and even the best testing often fails to capture critical issues. Complexity makes it difficult to proactively find problems before they are shipped, resulting in rework.
Companies spend billions simply to understand the ecosystem of tools, technologies, libraries, and code that produce modern software applications. Observability and application performance monitoring of production software systems are well-established devops and IT practices. However, observations of poor performance measured in a production system may be a trailing indicator of poor code quality. Estimates suggest between 40% and 90% of today’s production performance issues are at the code level. Performance issues can take days or weeks to resolve if they escape to production systems.
Unfortunately, the information developers need to test their applications for code-related performance issues is absent from the code-writing experience. Observability tools often rely on “load,” making the application of these tools in testing and staging environments unsuitable for performing these kinds of analyses.
Furthermore, the code patterns causing performance issues are often complex. Without knowing “where to look” in the code base, searching for these issues, even with tracing or debugging data, is next to impossible.
Runtime security analysis is missing
Identifying security flaws in code is an ongoing battle of security scanning, bug bounties, red teams, pen testing, and other techniques. Despite billions of dollars of investment in cybersecurity tooling, the problems of code-related security flaws persist, and the turnaround time for fixes is slow. MITRE’s Common Weaknesses Enumeration and OWASP’s Top 10 highlight the rise of complex code-design problems, such as broken access control, as crucial areas of software vulnerability. Unfortunately, these issues are not appropriate for static application security testing tools. The lack of visibility into the code’s runtime behavior results in an application security testing gap.
Missed opportunities to identify and remediate security flaws early in the process negatively impact the developer experience (DX). Secure code is a fundamental expectation that every developer wants to meet. Yet, in a complex service environment, maintaining a solid code security posture and delivering timely new functionality puts a cognitive tax on every line of code written.
Closing the gap with developer-centric observability
The software development process from the perspective of the software developer is ripe for improvement. An emerging category of tools is stepping up to unleash developer creativity and accelerate code delivery to drive businesses forward.
AppMap was built on the simple premise that developers should be able to see the behavior of software as they write code. Dynamic runtime information about software during the coding process is a critical information gap that needs to be addressed so developers can make better coding decisions. Runtime data during the coding process has a host of benefits for individual developers and development teams.
AppMap collects information about how code works when it runs, for any version of the code for any environment: the latest code deployed to production, in a pull request, or in the most recent uncommitted changes a developer is working on. AppMap helps developers understand the code they are creating by recording and visualizing software’s internal behavior as developers write. AppMap data includes libraries, APIs, web services, data connections, I/O, security modules, timing data, and more. This information builds a shared understanding of the software, using objective data from the code itself, with a new runtime perspective.
Bringing runtime analysis to the code editor
AppMap plugs into the developer’s code editor (Visual Studio Code or JetBrains IDEs) and provides suggestions for runtime optimizations. Problem areas are highlighted in the editor to call attention to the relevant code that can be improved. By delivering predictive and actionable performance and security analysis directly within the code editor, AppMap helps developers make improvements as code is written, avoiding cycles of rework. Before the code leaves their code editor, developers can see and evaluate slow API requests, repeated SQL queries, and other common code-related performance issues.
Not only does AppMap work much faster than traditional DAST and IAST tools, but it is able to be definitive in its findings regarding code fixes. Legacy tools might find an attack vector in your application, but they don’t help find the exact lines of code where the vulnerability exists. AppMap reduces the burden of reproducibility to pinpoint the specific parts of the application—right down to the lines of code—that need addressing, and why. The AppMap open source community is working to define more best practices for runtime software analysis in the code editor.
Performance and security are just two examples of the runtime software flaws that AppMap finds automatically during the coding process. Additionally, automated generation of OpenAPI documentation (formerly SwaggerDoc) is a core feature, given that so many of today’s applications are built to be API-first services. Keeping API documentation up-to-date is time-consuming for developers. And like many other forms of documentation, API docs quickly go out of date as the code evolves.
AppMap supports the most popular programming languages and frameworks, including Java, JavaScript, Python, and Ruby, with more being added soon. It is deeply integrated with code editors and developer tools, so using it is a natural extension of what developers do every day.
Harness the creative power of software developers
Creating new products for customers is at the heart of what drives developers’ love for coding. Companies that rely on developers for their success need to prioritize developer experiences and reduce toil. Engineering leadership should pay close attention to the explosion of novel tools and approaches like autonomous software development and generative coding tools that work in the coding environment, like GitHub Copilot.
But relying on the code base as the sole source of optimization is not enough. It is time to move the developer experience beyond the static, to include runtime data and new insights during the coding process to improve development, design, and software quality. AppMap complements all of these approaches by adding new data to the development experience: dynamic runtime data to improve the efficacy of code.
2023 is the year of developer experience. Embrace the opportunity to harness the full power and creativity of your software development teams with new approaches and technologies to minimize developer toil and deliver successful applications.
Elizabeth Lawler is founder and CEO of AppMap.
—
New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.