Fletcher: A framework to integrate FPGA accelerators with Apache Arrow
Fletcher is a framework that helps to integrate FPGA accelerators with tools and
frameworks that use Apache Arrow in their back-ends.
Apache Arrow specifies an in-memory format
targeting large datasets and provides libraries for various languages to
interface with the data in that format. Arrow prevents the need for
serialization between different language run-times and provides zero-copy
inter-process communication of datasets. Languages that have Arrow libraries
(under development) include C, C++, Go, Java, JavaScript, Python, Ruby and Rust.
While many software projects can benefit from these advantages, hardware
accelerated applications have also seen serious serialization bottlenecks.
Fletcher focuses on FPGA accelerators. Through Fletcher and Arrow, interfacing
efficiently between FPGA accelerator and high-level language runtimes is made
available to all the supported languages.
Given a set of Arrow
Schemas, Fletcher
generates the following:
Fletcher currently supports reading/writing from/to multiple Arrow
RecordBatches with an Arrow Schema created from any (nested) combination of:
In the future we would like to support:
Arrow::Table
)Tested simulators include the free and open-source
GHDL and the proprietary Mentor Graphics
Questa/Modelsim, and Xilinx Vivado XSIM.
The following platforms are (partially) supported (may be work-in-progress):
Our framework is functional, but experimental.
Especially the development branch (which is currently our main branch) may break
without notice. Some larger examples and the supported platforms are quite hard
to integrate in a CI pipeline (they would take multiple days to complete and
would incur significant costs for platforms such as Amazon’s EC F1). For now,
these larger examples and platform support resides in separate repositories
(shown above) and are tested against a specific tag of this repository.
Tutorials:
Hardware design flow:
Software design flow:
External projects using Fletcher:
If you used or studied Fletcher, please cite:
Additional publications: