Apple has significantly upgraded its local AI capabilities in the A20 Pro. The new iPhone 18 Pro is capable of running the Bonsai 27B model at roughly twice the speed of its predecessor, although the impressive results are limited by a rather prosaic limitation – the amount of RAM.
The main innovation of the A20 Pro is the Neural Engine with two 16-cores. Paired with 12GB of faster LPDDR5X and 115.2GB/s memory bandwidth, this allows iPhone 18 Pro to significantly speed up token generation. Adrien Grondin gave a demo of the Bonsai 27B, showing how quickly a smartphone can process a model with 27 billion parameters without connecting to a cloud server.
But there is a nuance. The newer Bonsai 2 uses 2-bit quantization and is already too large to run comfortably natively on the iPhone 18 Pro. Performance dropped significantly due to insufficient memory. Previous versions of Bonsai with 1-bit quantization were much less demanding and worked even on devices with 4 GB of RAM.
Ultimately, the A20 Pro does turn the iPhone into a more important native AI platform, but the processing power itself is only half the story. Even the most advanced neural engines cannot force models to fit into memory, which is physically insufficient.
