Enhancing Raspberry Pi with the Co-Pi-lot AI Kit: A Cost-Effective Solution for Machine Vision

The speaker built a custom Co-Pi-lot, a Raspberry Pi AI PC, with 13 TOPS of neural compute power using a $70 Raspberry Pi AI kit that includes an M.2 HAT and a 13 TOPS Hailo NPU, providing faster processing than Apple's M4 or Snapdragon X.

The Hailo-8L NPU runs at 13 TOPS and 8 TOPS per watt efficiency, offering a modern and improved version compared to the Coral TPU, which costs $60 for 2 TOPS and achieves around 2 TOPS per watt efficiency.

The AI kit, priced at $70, is focused on machine learning and neural computing, with practical applications like machine vision, object detection, pose estimation, and image segmentation, particularly emphasizing the use of Pi cameras for real-world problem-solving scenarios.

Various industries such as traffic planning, factory monitoring, agriculture, and individual experiments are utilizing AI for solving real-world problems like saving energy by recording clips only when people are detected, counting cars on highways, monitoring production lines, detecting spoilage, conducting experiments with automated microscopes, robotics, and safety monitoring.

Different approaches to AI chip integration include companies like Apple, Qualcomm, and Rockchip building custom NPUs into their main chips, edge devices like Raspberry Pi allowing for AI add-ons, and Arm advocating for AI extensions directly on the CPU, each approach having its benefits based on factors like power consumption, processing capabilities, and specific use cases.

While GPUs excel in training and running AI models, they often have higher power requirements and can be challenging to work with for low-power edge devices. In contrast, NPUs like the Hailo offer efficient processing suitable for devices with limited power budgets, such as the Raspberry Pi AI kit, which avoids the high power consumption associated with GPUs.

The Raspberry Pi 5 offers limited RAM, restricting the models that can be run on it. Training models on the Pi 5 is significantly slower compared to modern GPUs, taking days instead of minutes or hours.

Running YOLOv5, an object identification model, on the Pi 5's CPU without the AI kit results in slow performance at 2-3 frames per second. In contrast, the same model on the Hailo accelerator shows real-time object identification with minimal CPU usage and power consumption.

Utilizing AI accelerators like Hailo results in faster processing, lower power consumption, and frees up the CPU for other tasks. This enables real-time pose estimation, gesture-based apps, behavior prediction, image segmentation for effects, and monitoring multiple feeds efficiently.

Hailo provides a model zoo with pre-built models for upscaling and face recognition. Raspberry Pi is developing documentation for integration, enhancing support for projects. Future integrations like Frigate and StereoPi are anticipated, expanding the platform's capabilities.

Raspberry Pi may not excel in tasks like image generation or running Large Language Models due to limited RAM. While NPUs like SophGo's TPU could support some LLMs, Raspberry Pi's capabilities may be constrained for such tasks.

To surpass Microsoft's 13 TOPS of neural compute, the speaker plans to test multiple external PCI Express devices with varying TOPS. The setup involves Pineboard's HatBrick! Commander, Hailo accelerators, Coral TPUs, and PCI Express switches to maximize neural compute performance.

The speaker attempts to plug in a device to check for power, noting differences in pinouts on the Pi's M.2 hat. They mention three Coral Edge devices, one Hailo 8, and the absence of the 8L variant. Subsequently, they introduce the HatDrive Top from Pineboards, creating a configuration with multiple Pineberry boards, which is described as an unsupported setup.

The speaker observes that all connected devices are receiving power without issues, expressing relief at the absence of any malfunctions or 'magic smoke'. They proceed to run 'lspci' to check the connected hardware, identifying various components such as dual edge TPU, Hailo 8, Hailo 8L, and a Coral TPU.

The speaker encounters a challenge with booting up the Hailo device, attributing it to a potential power requirement issue. They speculate that the high power draw from multiple connected devices may be causing a brownout situation, leading to the Hailo board not being ready for operation.

The speaker suggests opting for a more powerful NPU from Hailo's lineup if higher performance is needed, mentioning models with 52 to 208 TOPS starting at $250. They advise against the current setup due to power constraints and recommend exploring alternatives. Additionally, they highlight the evolving landscape of AI products and the potential for advancements in the field.

The speaker expresses their positive opinion on the AI kit, acknowledging its niche appeal for machine vision and robotics enthusiasts. They note that while the kit may not be essential for most users, it serves as a valuable alternative to the Coral platform, catering to specific needs within the AI development community.