In the post-Moore era where data is growing exponentially, AI technology supported by powerful computing power is booming, and the requirements for computing power are also increasing day by day.
As AI training and inference costs continue to rise, the number of parameters for LLM (Large Language Model) has grown from 94 million parameters in 2018 to commercially available 175 billion parameters for GPT-3, and GPT-4 is expected to exceed 1 trillion . Data shows that the computing power required to train an AI model will increase by 275 times every two years.
Advances in data processing technology have driven the evolution of computers, but traditional processing units and large computing clusters cannot break through the boundaries of computational complexity. Moore's Law, while still progressing and reborn, cannot explain the need for faster and more powerful computing power.
Nvidia CEO Jensen Huang once said frankly: Moore’s Law is dead. As computing power continues to push boundaries, where are the opportunities for AI?
Bank of America Merrill Lynch pointed out in an in-depth report released on March 21 that the next generation of computers will include: high-performance computing (HPC), edge computing, space computing, quantum computing and biological computing.
High Performance Computing (HPC)
High-performance computing refers to computing systems that use supercomputers and parallel computer clusters to solve advanced computing problems.
The report points out that high-performance computing systems are often more than 1 million times faster than the fastest desktop computers, laptops or server systems, and have a wide range of applications in established and emerging fields such as autonomous vehicles, the Internet of Things and precision agriculture.
Bank of America believes that the development trend of high-performance computing will bring room for growth in accelerators for ultra-large-scale systems (including LLM).
Although high-performance computing accounts for only a small part of the data center total available market (TAM) (about 5% share), the future development trend is to become a leading indicator for cloud/enterprise applications.
Especially as LLM's demand for computing power becomes higher and higher, 19 of the 48 new systems adopt accelerators, representing an accelerator attachment rate of approximately 40%. A survey of the world's top 500 companies shows that there is room for growth in the use of accelerators in ultra-large-scale service systems. Currently, only about 10% of servers are accelerated.
Bank of America pointed out that another trend is that with the help of co-processors (processors developed and applied to assist the CPU in completing processing tasks that it cannot perform or performs efficiently and ineffectively), computing methods will increasingly change from serial to serial. Turn to parallelism.
The maturation of Moore's Law/serial computing is shifting more workloads to parallel computing, enabled by the use of independent coprocessors/accelerators such as GPUs, custom silicons (ASICs), and programmable silicons (FPGAs) .
As of November 2023, 186 machines in the Global 500 use coprocessors, up from 137 systems five years ago; coprocessor/accelerator usage among the Global 500 is flat month-on-month, and year-over-year The total computing performance of the top 500 supercomputers increased to 7.0 exaflops, a year-on-year increase of 45%.
spatial computing
Spatial computing refers to computers that use AR/VR technology to integrate the user's graphical interface into the real physical world, thereby changing human-computer interaction.
In fact, we are reaching an inflection point in human-computer interaction: moving away from traditional keyboard and mouse configurations toward the edge of touch gestures, conversational AI, and enhanced visual computing interactions.
Bank of America believes that after PCs and smartphones, spatial computing has the potential to drive the next wave of disruptive changes - making technology a part of our daily behavior and connecting our physical and digital lives with real-time data and communications.
For example, Apple's Vision Pro has taken a crucial step.
edge computing
Compared with cloud computing, edge computing refers to processing data physically closer to the terminal device, which has more advantages in terms of latency, bandwidth, autonomy and privacy. According to research firm Omdia, an "edge" is a location where the round-trip time to the end user is at most 20 milliseconds (milliseconds).
Bank of America said many enterprises are investing in edge computing and edge locations (from internal IT and OT to external, remote sites) to get closer to end users and where data is generated.
Tech giants like Facebook, Amazon, Microsoft, Google and Apple are all investing in edge computing, and returns on this investment are expected to be a driver of stock performance for these companies over the next five years.
It is expected that by 2025, 75% of enterprise-generated data will be created and processed at the edge.
According to data from research organization IDC, the market size of edge computing is expected to reach US$404 billion by 2028, with a compound annual growth rate of 15% from 2022-28.
It is expected that the development trajectory of the edge computing market will be roughly as follows between 2022 and 2025:
Phase 1 (2022): Use Cases - Highly Customized; Phase 2 (2023): Vertical Domains - Vertical Suites/Packages; Phase 3 (2024): Horizontal Domains - Cross-Vertical Technologies; Phase 4 ( 2025): IT Strategy - Vertical Strategy.
In the future, Bank of America believes that AI opportunities will come from reasoning, and for edge computing reasoning, CPU will be the best choice.
Unlike training that takes place in core computing, inference requires a distributed, scalable, low-latency, low-cost model, which is exactly what edge computing models provide. The current divide in the edge computing industry is whether to use CPUs or GPUs to support edge inference. While all major vendors support both GPU and CPU capabilities, we believe that CPUs are the best choice to support edge inference.
Under the GPU model, only 6-8 requests can be processed at a time. However, the CPU is able to segment the server by user, making it a more efficient processing system at the edge. Instead, CPUs provide cost efficiency, scalability, and flexibility and allow edge computing vendors to overlay proprietary software on the computing process.
fog computing
In the field of edge computing, there is also an extended branch concept: fog computing.
Fog computing is a network architecture that uses terminal devices to store, communicate, and transmit data when performing large amounts of edge computing on-site.
Bank of America believes that fog computing and cloud computing are complementary and may form a hybrid/multi-cloud deployment format in the future.
As applications move to the cloud, hybrid/multi-cloud approaches are being deployed. Cloud computing and edge computing are complementary, and using a distributed approach can create value by solving different needs in different ways.
An IDC survey revealed that 42% of enterprise respondents have difficulty designing and implementing key components including infrastructure, connectivity, management and security. In the long term, the combination of edge data aggregation and analytics with the scaling capabilities of cloud access, such as analytics and model training, will create a new economy built on digital edge interactions.
Quantum computing
Quantum computing refers to computing that uses subatomic particles to store information and uses superposition to perform complex calculations.
Bank of America believes that the importance of quantum computing lies in its inherent and irreplaceable advantages in solving problems that cannot be solved by traditional computers - this is also known as "quantum supremacy". However, the commercialization process of quantum computing is still in its infancy.
Quantum computing can solve problems almost instantly that would take conventional computers billions of years to solve. We are in the very early stages of adoption, with only a few machines deployed on the cloud for commercial use, mainly for research. However, the commercialization process is proceeding rapidly.
Bank of America believes that quantum computers break the boundaries of computing, and the combination of the two most powerful technologies, AI and quantum computers, can fundamentally change the physical and mathematical worlds.
In the short and medium term, life sciences, chemicals, materials, finance and logistics industries will benefit the most. In the long term, when AI reaches human cognitive capabilities and even has self-awareness, general artificial intelligence (AGI) will lead to a fundamental change in technology.
The report states that quantum computers are not suitable for routine tasks like using the Internet, office tasks or email, but are suitable for complex big data calculations like blockchain, machine and deep learning or nuclear simulations. The combination of quantum computers and 6G mobile networks will change the rules of the game in all walks of life.
Big Data Analysis: Huge Untapped Big Data Potential The untapped big data potential is huge, and the amount of data created is expected to double by 2024, increasing from 120ZB in 2022 to 183ZB.
IDC data shows that currently, due to computing power bottlenecks, we only store, transmit and use 1% of the world's data. But quantum computing could change that and unlock real economic value – potentially using 24% of the world’s data and driving a doubling of global GDP.
Cybersecurity: With parallel processing capabilities of up to 1 trillion calculations per second (Bernard Marr), quantum computing is capable of technically challenging all current encryption methods, including blockchain. This also opens the door to new encryption technologies based on elements of quantum computing.
Artificial Intelligence and Machine Learning: Advances in machine learning and deep learning are limited by the speed of computation on the underlying data. Quantum computers can accelerate machine learning capabilities by solving connections between complex data points faster using more data.
Cloud: This is probably one of the winners because the cloud is probably the platform where all data creation, sharing, and storage happens. Once commercialization of quantum computers begins, cloud access will be required and data generation should grow exponentially. Therefore, cloud platforms will be the solution.
Autonomous vehicle fleet management: One connected autonomous vehicle will generate the same amount of data as 3,000 internet users; for two vehicles, the amount of data generated will jump to about 8,000-9,000 users. Therefore, the growth in data generated by autonomous vehicles alone will be exponential.
brain computer interface
Brain-computer interface refers to the direct interaction with the external world through the brain waves of humans and animals.
Bank of America pointed out that startups like Neuralink are studying human-machine cooperation through implants (BCI), and have implemented brainwave control devices in animal experiments, and early human clinical trials are still in progress.
Currently, brain-computer interface (BCI) and brain-brain interface (CBI) technologies are under development, and there are already examples of hand movements controlled through thoughts.
Synchron's solution is to place a grid of tubes lined with sensors and electrodes in the blood vessels that supply the brain, from which neuron signals can be received. After the signals are passed to an external unit, they will be translated and communicated to a computer. In clinical trials, paralyzed individuals were able to text and email, as well as bank and shop online.
Neural's implants include neural wires that are inserted into the brain via a neurosurgical robot to pick up nerve signals, allowing clinical patients to now move a computer mouse just by thinking.
