Skip to content

fix(native): Hoist plugin loading and de-inline factory map for dynamic connector support#156

Open
20001020ycx wants to merge 1 commit into
masterfrom
yscope/refactor/deinline-connector-registration
Open

fix(native): Hoist plugin loading and de-inline factory map for dynamic connector support#156
20001020ycx wants to merge 1 commit into
masterfrom
yscope/refactor/deinline-connector-registration

Conversation

@20001020ycx
Copy link
Copy Markdown

@20001020ycx 20001020ycx commented Apr 22, 2026

Description

  • Hoist registerDynamicFunctions() before registerVeloxConnectors() in PrestoServer.cpp
  • De-inline connector factory registration functions from Registration.h into Registration.cpp

Motivation and Context

This PR enables dynamic connector plugins — connectors loaded at runtime via plugin-dir using dlopen() — as described in RFC-0019: Connector Plugins. Two changes are needed to make this work correctly.

Both changes revolve around a singleton factory map defined in Registration.cpp:

static std::unordered_map<std::string, std::shared_ptr<ConnectorFactory>> factories;

This map lives inside detail::connectorFactories() and maps connector names (e.g. "hive", "tpch") to their ConnectorFactory instances. Both registerConnectorFactory(name, factory) and getConnectorFactory(name) read and write this map.

Why registerDynamicFunctions() must be hoisted before registerVeloxConnectors()

registerVeloxConnectors() iterates over every *.properties file in the catalog/ directory. For each one, it reads connector.name and calls getConnectorFactory(connectorName) to look up the factory in the singleton map, then calls factory->newConnector(...) to instantiate the connector.

registerDynamicFunctions() loads plugin .so files from the plugin/ directory via dlopen(). Each plugin's registerExtensions() entry point calls registerConnectorFactory(...) to insert its factory into the same singleton map.

Before this PR, registerVeloxConnectors() ran before registerDynamicFunctions(). Since plugins had not been loaded yet, the factory map had no entry for dynamically registered connector names. When registerVeloxConnectors() processed a catalog .properties file referencing a dynamic connector, getConnectorFactory() threw because the corresponding factory had not been registered yet.

So the sequence must be:

  1. registerDynamicFunctions() — loads plugin .so files, plugins call registerConnectorFactory() to insert into the map
  2. registerVeloxConnectors() — reads catalog/*.properties, calls getConnectorFactory() to look up entries in the map

The original code had no problem with built-in connectors (Hive, TPC-H, etc.) because their factories are registered statically at compile time.

Why registration functions must be de-inlined

Before this PR, the factory map was a static local variable inside the inline function connectorFactories() in Registration.h. Any connector shared library that #includes this header gets its own copy of the static variable, meaning both presto_server and the plugin library hold separate factory maps (Note, shared library references the header file in the presto-native-execution for compilation, the implementation are resolved at run time):

presto_server           → factory map at 0xAAAA  {"hive": ..., "tpch": ...}
libpresto_plugin.so     → factory map at 0xBBBB  {"my_connector": ...}

This breaks the singleton property of the factory map: the plugin's registerConnectorFactory() inserts into its own map (0xBBBB), but the server's getConnectorFactory() reads from its own map (0xAAAA) — and finds nothing.

De-inlining connectorFactories() ensures the static local variable exists in exactly one place (Registration.cpp, linked into presto_server), allowing dynamically registered connector known to presto_server successfully.

Test Plan

  • Build presto_server with the changes
  • End-to-end tested with a customized native connector shared library (.so) loaded via plugin-dir, verifying successful connector registration and query execution.

Release Notes

== NO RELEASE NOTE ==

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 22, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9c22c7d9-1dcf-45bb-8aad-b168256f622e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch yscope/refactor/deinline-connector-registration

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@20001020ycx 20001020ycx force-pushed the yscope/refactor/deinline-connector-registration branch from d8e4993 to f7d3730 Compare April 22, 2026 20:41
@20001020ycx 20001020ycx changed the base branch from release-0.293-clp-connector to master April 22, 2026 20:47
@20001020ycx 20001020ycx force-pushed the yscope/refactor/deinline-connector-registration branch 3 times, most recently from 62db030 to abf1515 Compare April 23, 2026 15:11
@20001020ycx 20001020ycx changed the title refactor(native): De-inline connector factory registration for dynamic plugin support fix(native): Hoist plugin loading and de-inline factory map for dynamic connector support Apr 23, 2026
@20001020ycx 20001020ycx force-pushed the yscope/refactor/deinline-connector-registration branch from abf1515 to 8e60013 Compare April 23, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant