Microsoft uses AI to expedite pull requests – the feature that developers can use to notify others of changes to code – to remind authors to deal with their overdue requests. This comes from a new whitepaper released this week that describes Nudge, an end-to-end service that uses models based on effort estimation to predict the completion time for a given pull request. According to Microsoft, Nudge has been deployed to 147 internal repositories since 2019 and “significantly” reduced the completion time for the 8,500 pull requests for which notifications were sent by 60%.
With the advent of platforms like GitHub and GitLab, pull requests have become the standard mechanism for distributed code reviews. They allow one or more developers and even bots to review changes. Once the reviewers have logged out, the changes can be merged with the main branch and posted. Pull requests streamline the review process, but they can also slow down the software development process. For example, if reviewers become overloaded and lose track of the requirement, it may not be reviewed in a timely manner.
While looking for a solution, Microsoft and Dell researchers developed Nudge that uses effort estimation and activity detection algorithms to predict the completion time for a given pull request. Nudge goes a step further, analyzing the request status to identify the author or reviewers of the pull request holding the request from completion and notifying them of comments in the pull request environment.
To create nudge, the researchers performed a correlation analysis to analyze various factors related to pull requests, pull request authors, pull request reviewers, and even timing factors that affect the request completion time. They then built the cost estimation model to predict the completion time of the pull request at the time the request was created, using metrics such as code churn, reviewer information, and ownership information. Next, they implemented activity detection to monitor any updates to the pull request (such as new commits and review comments) and adjust notifications accordingly. To make the nudge notifications actionable, the researchers used what is known as a dependency determination that identifies the author of the pull request or a specific reviewer.
After deploying nudge to repositories at Microsoft for 18 months, the researchers report that 73% of pull request notifications were “positively acknowledged” by users. “Pull requests are an integral part of the collaborative software development process,” they write. “Only 1,570 pull requests out of 8,500 pull requests (18.47%) took more than a week to close. 81.53% of pull requests are closed within a week. One important note is that 2,300 pull requests (27.05%) of the pull requests that Nudge sent notification on were completed in one day. This distribution shows that most pull requests that Nudge sends notifications on complete relatively quickly. “
Nudge builds on Microsoft’s earlier efforts to infuse the DevOps process with AI and machine learning. For example, in April the company described in detail a system that, according to its own information, correctly differentiates between security and non-security software errors in 99% of cases and that precisely identifies critical security errors with high priority in 97% of cases. Microsoft uses AI to troubleshoot cloud service outages.