UMVA has learned that a groundbreaking solution is being developed to overcome the significant hurdle of running sophisticated AI models on personal computers.
The challenge lies in the massive memory and storage requirements of these models, which often exceed the capabilities of most PCs. However, a collaboration between Phison and Intel is working to change that.
Using Phison's innovative aiDAPTIV solution, a 26 billion parameter AI model can now be run on a laptop with a mere 16GB of RAM, a significant reduction from the 32GB normally required. This breakthrough has two major advantages: enabling local AI on more laptops and allowing more powerful laptops to load even larger models or run multiple tasks alongside AI.
Running AI on a PC can be a resource-intensive process, often monopolizing system resources and preventing other work from being done. This sometimes forces users to purchase a dedicated AI PC, a costly and limiting solution.
The problem lies in the way AI functions are processed, typically in video RAM or system RAM shared with the GPU. As AI models generate tokens on the fly, they require more and more memory, causing the system to bog down.
Phison's aiDAPTIV solution addresses this issue by using high-performance, extreme-endurance NAND flash as an AI cache, storing tokens to be recalled later. This intelligent technology anticipates the model's needs, seamlessly sending data between RAM and the SSD to enable larger models to run without impacting performance.
Phison claims that its aiDAPTIV technology can boost the response times of local AI to near-instantaneous levels, even as the context window grows. This is a significant improvement, making local AI more practical and efficient.
The collaboration between Phison and Intel focuses on enabling Phison's technology on Intel AI PC platforms powered by Intel Core Ultra processors. The two companies are working to demonstrate the technology for software vendors, which could eventually optimize their own apps for the technology.
However, there is a potential catch: the joint work is being performed using Phison's specialized Pascari AI100E SSDs, designed for high endurance and sustained performance. A successful implementation may require laptop makers to specifically purchase these SSDs, which could be a costly and restrictive requirement.
The success of this technology remains to be seen, and it is unclear whether users will adopt it widely. The outcome will depend on various factors, including the cost and availability of the specialized SSDs and the level of support from software vendors.
