IDSIA: LSTM History, Vanishing Gradients & Deep Learning

The Lugano institute's AI history covers Hochreiter and Schmidhuber's 1997 LSTM work, GPU-trained vision systems and highway networks.

kindResearch lab
founded1988
Event history

A philanthropist's institute in Lugano

IDSIA, the Dalle Molle Institute for Artificial Intelligence, was established in Lugano in 1988 by Angelo Dalle Molle, an Italian industrialist and philanthropist who wanted a research institute free of typical university bureaucracy. Since 2000 it has been jointly run by USI, the Università della Svizzera Italiana, and SUPSI, the University of Applied Sciences and Arts of Southern Switzerland, functioning as a bridge between the two institutions. It teaches into both schools' AI and data science programs and researches deep neural networks, robotics, machine vision, and AI methods for operations research, alongside work on algorithmic transparency and fairness. [1]

Andrea Emilio Rizzoli, appointed director in December 2020, described the institute in a 2021 interview as small by design, built to move faster on applied and fundamental research than a conventional university department could. [2]

A thesis that became a paper that became an era

Jürgen Schmidhuber joined IDSIA as scientific director in 1995 and supervised Sepp Hochreiter's earlier 1991 diploma thesis on what is now called the vanishing gradient problem: in a deep or recurrent network, the training signal shrinks as it propagates backward through many layers, so a network cannot learn dependencies stretching more than a few steps back. Schmidhuber has called that thesis one of the most important documents in the history of machine learning. [3]

Hochreiter and Schmidhuber's solution, published as Long Short-Term Memory in November 1997, introduced a gated memory cell that could hold a signal across long sequences without it vanishing or exploding. Felix Gers and colleagues refined the architecture around 2000, and the resulting design ran most of the field's sequence modeling work, machine translation, speech recognition, and text generation among them, until attention-based architectures took over that role in the late 2010s. [4][3]

Not a one-paper institute

LSTM was IDSIA's most consequential export, but not its only one. Around 2011, a team including postdoctoral researcher Dan Ciresan built GPU-accelerated convolutional neural networks that won multiple computer vision competitions and helped demonstrate that raw compute, applied to the right architecture, could substantially beat prior approaches, a result that predates and anticipated the GPU-driven deep learning boom of the following decade. In 2015, Schmidhuber worked with Rupesh Kumar Srivastava and Klaus Greff on highway networks, a feedforward architecture that made it practical to train networks hundreds of layers deep, a precursor to the residual connections that became standard in later architectures. [3]

Atlas interpretation: IDSIA is a small institute, not a lab with a consumer product or a funding round to track, which is why it shows up on the timeline through a single paper rather than a string of company news. That paper's staying power, two decades as the default way to model sequences, is itself the case for including a fifty-person research institute in a history mostly populated by companies with market capitalizations. [4]

Sources

  1. Overview - IDSIA

    IDSIA

  2. Andrea Emilio Rizzoli - Director, Dalle Molle Institute for Artificial Intelligence (IDSIA/USI-SUPSI), Switzerland

    Pharmaboardroom · Mar 25, 2021

  3. Jürgen Schmidhuber

    Wikipedia · Sep 9, 2026

  4. Long Short-Term Memory

    Neural Computation · Nov 15, 1997