Tiny Corp. Reveals Details About "TinyBox" System
Tiny Corp., a specialist in neural network frameworks, has shared insights into the ongoing development of its "tinybox" system. According to the company, transparency is key in their approach to building the system. They have the components to construct 12 boxes, with a nearly final case design. Overcoming PCI-E AER errors during the process was challenging, but the team persevered. The Bill of Materials (BOM) cost for each box is approximately $10k, with a selling price of $15k. The founder, George Hotz, emphasized the year-long engineering effort put into the project, highlighting the complexity of the task. Despite initial skepticism, Hotz believes that customers are getting a great deal for $15k, especially for those not building in large quantities.
One notable feature of the TinyBox system is the integration of Team Red's flagship gaming GPU, a departure from Hotz's past rejection of NVIDIA AI hardware. The system will not utilize AMD's latest Instinct MI300X accelerators, opting instead for RDNA 3.0 over CDNA 3.0, likely due to the increasing demand for enterprise-grade GPUs in the industry.
The TinyBox build consists of a rack-mounted 12U chassis housing an AMD EPYC 7532 processor with 128 GB of system memory. Storage is handled by five 1 TB SN850X SSDs (4 in raid, 1 for boot), with an empty 16x OCP 3.0 slot reserved for networking tasks. Two 1600 W PSUs provide the necessary power for the system. Each TinyBox system is equipped with six XFX Speedster MERC310 RX 7900 XTX graphics cards.
Hotz's vision for the Tiny Corp. is ambitious, aiming to commoditize the petaflop with the help of tinygrad. The system image shipping with the box will be Ubuntu 22.04, featuring tinygrad out of the box. Hotz expressed confidence in the progress of PyTorch and JAX support on AMD hardware, emphasizing the flexibility for users to customize their hardware. With 583 preorders already in place, the company plans to order parts for 100 more sets next week, representing a $1M investment. The production line has already begun running, with shipments expected to start in April.
