The Promise of Analog AI
Why Analog AI is About to be in Everything from Alexa to Your iPhone
Neural networks keep getting larger and more energy-intensive. As a result, the future of AI depends on making AI run more efficiently and on smaller devices.
That’s why it’s alarming that progress is slowing on making AI more efficient.
The most resource-intensive aspect of AI is data transfer. Transferring data often takes more time and power than actually computing with it. To tackle this, popular approaches today include reducing the distance that data needs to travel and the data size. There is a limit to how small we can make chips, so minimizing distance can only do so much. Similarly, reducing data precision works to a point but then starts to hurt performance.
We’ve gone from 32- and 64-bit floating-point numbers to . The speedups were dramatic, but lower precision will affect accuracy. Other approaches are being explored, but the limits of digital are starting to be hit.
Decades ago, digital computers trumped analog computers due to the need for precision. Just like reducing data precision from 32-bits to 8-bits did not impact accuracy in learning, we are finding that precision is less important for AI. It’s time for the comeback of analog. Researchers and industry are racing to meet the challenge, with IBM recently announcing and via analog AI. How’s this possible?
The Von Neumann Bottleneck
In computing, the von Neumann bottleneck describes the phenomenon that system throughput is limited by the relative speed of data processing versus movement. If we could store and process the data in the same place, we would eliminate data movement, removing this bottleneck.
With analog AI, data is stored and processed in the same place. Using non-volatile memory devices, analog can store data without power for .
At the same time, the data needs fewer transistors than digital circuitry. In digital machines, you have 1s and 0s. To represent a number in the range -128 to 127, you’d need 8 bits. To add two numbers you’d need approximately 200 transistors.
Unlike digital, analog is not binary. It can represent a whole range of numbers based on the conductance or resistance (measure of electrical current). Normally, digital has a cut-off threshold for 0s and 1s, but analog uses the actual value. As a result, you can represent many numbers in one transistor, as long as you’re okay with the variability that results naturally from the transistor’s physical properties and randomness. This is problematic in traditional computing but workable in deep learning.
How Does Analog AI Process Data?
In neural networks, the most common operator is multiply-accumulate. You multiply sets of numbers and then sum them up, as used in matrix multiplication that’s the backbone of deep learning. If you store the inputs as arrays, you can actually do this in one swoop by utilizing physical engineering laws ( to multiply, to sum) on a full matrix in parallel.
This is the crux of analog AI. If it was that easy, analog AI would already be used. Why aren’t we using analog AI yet?
The Problems of Variability
Conductance can vary based on everything from the manufacturing of the chip to environmental factors. The variation can add up and throw neural networks off.
Early approaches used analog and digital together, with digital-to-analog and analog-to-digital convertors between layers. However, this needs to be limited, as conversion is slower and more energy-intensive than staying analog. It can create its own bottleneck.
Analog is also not ideal for high-precision applications, where every bit matters. Ditto for applications where explainability is important: if you want to understand why your neural network made a certain prediction, introducing variability makes explaining it more challenging. Your network isn’t deterministic. Given the same data, variability may lead the network to a different prediction next time.
That’s why analog AI is best-suited now for AI on the edge devices, like your phone and Google Home. Your battery would last much longer, your energy bill would go down, and you can even get a smaller device (fewer transistors).
The barriers to wider adoption are being worked on by researchers. What’s next?
The Future
Right now, analog AI works successfully for multiply-accumulate operations. For other operations, it is still ideal to provide their own circuitry, as programming nonvolatile memory devices takes longer and results in faster wear and tear than traditional devices.
Inference does not typically require reprogramming these devices, since weights rarely change. For training, however, they would require constant reconfiguration.
In addition, analog’s variability results in a mismatch between error in forward propagation (inference) and backpropogation (calculating error during training). This can cause issues during training. That leads to a need for separate circuits for backpropogation, likely digital. However, researchers are already coming up with ways to train on the same analog AI neural networks so that they are end-to-end. For example, a team from the University of Montreal and Rain Neurophormics to use in end-to-end chips.
The Takeaway
As ways to make neural networks faster and more energy-efficient achieve diminishing returns, analog will likely make a comeback. Neural networks have lower precision needs than traditional computing, making analog possible.
With the ability to produce networks that are many times faster, smaller, and more energy-efficient, analog AI is becoming a hot research area and one that has enormous commercial potential. Expect to see analog circuitry in your smartwatch, phone, and home assistants soon!
