IMLane: Composable Framework for Efficient AI Function Execution in Database Engine

IMLane is a high-performance, standalone, composable development framework written in C++ 17. It is designed to suppport scalable Python-UDF based prediction query execution within database engines.

Project Structure

• assets/: Contains project-related assets.
• cmake/: CMake configuration files for project setup.
• contrib/: Third-party contributions and extensions.
• database/: Databases inegrated with IMLane, OceanBase (based on IMBridge), DuckDB as integration examples.
• include/: Header files for the project.
• src/: Source code for the core framework.
• test/: Unit tests and integration tests.
• third_party/: External dependencies and libraries.

Architecture

The diagram above highlights the high-level architecture of IMLane, emphasizing its modular design and seamless integration with database engines.

IMLane consists of three components: the DBEnd Library, the Coordinator, and the Backend Executors. Specifically, the DBEnd library serves as the main entry point provided to database engine developers. It is designed as a plugin that can be compiled and linked into existing database engines, like DuckDB and OceanBase. This library includes a set of DBEnd interfaces, i.e., data conversion interface and scheduling primitive, for developers to implement and call within the database engine code. The DBEnd Runtime utilizes scheduling primitives and implemented data conversion interfaces to interact with the Coordinator and drive the scheduling of AI functions.

The Coordinator serves as the core element of the IMLane framework. It includes the AI Function Scheduler, which handles scheduling requests from the DBEnd Runtime, and the Lane Manager, which manages the life cycle of Lanes containing the inter-process shared resources and compute resource units.

In addition, each Backend Executor embeds an execution runtime, e.g., a Python interpreter, in its corresponding process, which executes the specified AI function code and returns the execution result to the database engine via a Lane.

Getting Started

Prerequisites

• CMake (version 3.10 or higher)
• A C++ compiler with C++17 support
• Python 3.8+
• Ray 2.10.0+

Installation

1. Install dependencies:

   ./install_deps.sh
   cd contrib/pythonpkg
   pip install .

2. Build the project:

   mkdir build && cd build
   cmake .. -DCMAKE_INSTALL_PREFIX=/home/dist # install dir
   cmake .. -DBUILD_IMLANE_SERVER=ON -DCMAKE_INSTALL_PREFIX=/home/dist # build whole project
   cmake .. -DBUILD_IMLANE_SERVER=OFF -DCMAKE_INSTALL_PREFIX=/home/dist # or build dbend only
   make -j$(nproc) install
   /home/dist/bin/imlane --help # print help info of imlane coordinator

3. Built Framework Structure

• bin/: Executables, including the imlane coordinator binary.
• lib/: Shared libraries required for the framework.
• include/: Header files for DBEnd usage.

4. Used in your Project See the example in demo/cpp_example for how to integrate IMLane into your project.

License

This project is licensed under the terms of the LICENSE file.