Will Generative AI Drastically Change the World? -The Evolution Required for AI Services and the Capacitors Supporting Them - (First Part)

Mechanism of Generative AI

How does generative AI generate new content? This is accomplished roughly through the processes of (1) learning and (2) inference. (Figure 2)

(1) Learning
This is the process performed to build the model to execute the generation. During this process, calculations are performed to capture features of the data using a vast amount of data. As a result, an inference model is constructed to achieve advanced generation. The higher the degree of coverage/accuracy that the service handles, the longer it takes as a trade-off (sometimes several months). In order to launch and update services quickly, learning time is required to be reduced.

(2) Inference
This is the process of executing the generation (actual service). In this process, responses to user input are generated based on inference models obtained through learning. The content generated during this process is the content deemed to have the highest probability of matching the input. Inference is performed in far less time than learning (a few dozen seconds at most) by using inference models. Services with long response times are avoided by users, so it is required that they run as fast as possible, even by a second.

Issues with Generative AI

While generative AI is a highly promising technology due to its wide range of applications and convenience, some practical issues still remain. That is, as mentioned above, because the response content is generated solely on the basis of probability, there are cases where inaccurate content is generated. This phenomenon is called a "hallucination" because the AI outputs plausible lies as if it were hallucinating. What measures are effective to address these issues? One of the more effective methods recognized is to increase the amount of data and the number of computational parameters during the learning process. It has been publicly reported that the accuracy of the generative AI model used in OpenAI's ChatGPT has been improved by using this approach during the evolutionary process.

Hardware Supporting Generative AI

Currently, ChatGPT and many other generative AI services are provided as web services. Computer processing for executing web services is performed on a large number of servers installed in so-called cloud data centers.

In conventional web services, a single server was usually equipped with one or two CPUs*1 responsible for the main computational processes and only the CPUs were used to perform most processes when executing services. However, during the 2010s, the effect of circuit miniaturization, which was a main factor of improving CPU performance, became less significant and efficiently improving performance became difficult. During this situation, an approach to use accelerators (coprocessor) was thought of. The idea of this approach is to add a different type of high-efficiency processor to a server in order to efficiently improve the processing performance. Use of this system made application of AI that requires huge amount of data processing for web services a reality. As an accelerator for generative AI, GPU*2 or ASIC*3 is selected in general (see Note).

Market and Equipment Trends

Regarding learning and inference processing to improve AI performance, demand for improvement in data processing performance of hardware never stops. This is because both higher accuracy services and shorter lead times to provide them are important factors for companies that develop services using generative AI.

For this reason, generative AI processors have extremely high-end specifications and the number of installed units is increasing. As a typical configuration, a combination of two CPUs and eight accelerators is often seen. In newer hardware, an attempt for higher performance has started. The number of CPUs is increased to eight and both CPUs and accelerators are installed closely or configured within the same processor package in order to improve communication speed between them as well as power efficiency.

There are other approaches for improvement. Examples are the use of mega ASIC for higher learning speeds and implementation of AI processing functions on the processors of mobile devices and embedded devices on the terminal side in order to increase the speed of inference. News of new cases like these are being reported constantly. In any case, an important key for generative AI hardware is the improvement of processor performance utilizing cutting-edge technology and the optimum combination in the system overall. As generative AI rapidly develops, reduction in environmental burden and costs have become extremely important issues regarding hardware as well as software. Not only major companies but also many up and coming manufacturers have launched in this field.

Configuration examples of mainly processors that take a central role in hardware that supports generative AI have been introduced above. In order to operate such high-performance processors in a stable manner, electronic parts used with them are also required to meet high requirements. In the second part, performance required of capacitors that significantly contribute to the power supply of processors will be introduced together with specific solution examples of Panasonic Industry.