Microsoft Research: ResNet, Residual Learning & Lab History

A research lab with its own budget

Microsoft Research is Microsoft's research division, founded in 1991, and it operates separately from Microsoft's product groups: it publishes papers, releases open source projects and datasets, and pursues work with no committed shipping date, which is a different mandate than the product teams building Azure or Microsoft 365 Copilot. Peter Lee is president of Microsoft Research. The division runs labs across several countries, including Redmond, Cambridge in the UK, Montreal, New York City, India, and an Africa lab in Nairobi, alongside mission-focused groups such as AI for Science and AI Frontiers. [1]

That structure is the reason Microsoft Research has its own page here rather than being folded into Microsoft's. Microsoft the company is a customer of and investor in outside AI labs, principally OpenAI. Microsoft Research is Microsoft's own laboratory, and its output, papers and released architectures, is a different kind of contribution than a partnership or an equity stake. [1]

The shortcut that made depth trainable

In December 2015, Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun, working at Microsoft Research Asia, published Deep Residual Learning for Image Recognition, introducing what became known as ResNet. Image classification networks had been getting worse, not better, once they grew past roughly twenty layers, a failure distinct from overfitting that the paper calls the degradation problem: adding more layers made the training error itself rise. The paper's fix was to have each block learn a residual function, the difference between its input and its target output, and add that correction back onto the input, rather than learning the full transformation from scratch. With that change, networks that had been unstable past twenty layers trained cleanly at over a hundred, and a 152-layer version won the 2015 ImageNet Large Scale Visual Recognition Challenge. [2][3]

Atlas interpretation: The residual, or skip, connection outlived the image classifiers it was built for. It is a structural piece of the transformer architecture behind essentially every large language model on the timeline, including the ones with no other resemblance to a 2015 vision network, because it solves the same problem at any depth: it gives gradients a direct path backward through a deep stack of layers during training. A researcher publishing on a fixed benchmark left behind a component that the rest of the field kept using once the benchmark itself stopped mattering. [2]

One paper among decades of them

ResNet is Microsoft Research's only event on the timeline, but it is not the lab's only relevant output. Microsoft Research describes having published tens of thousands of papers and released hundreds of open source projects and datasets since 1991, spanning computer vision, human language technologies, systems, security, quantum computing and fields with no direct connection to AI, such as ecology and economics. The bar for a place on the timeline is deliberately high, so a single paper standing in for three decades of a lab's output says more about that bar than it does about the lab's total contribution to the field. [1]