set PG_SYSROOT instead of -isysroot

[tureba]

When -isysroot is added to CFLAGS, anyone who tries to compile extensions against the server headers will have to use the same SDK version and location as the server had, which may not be possible.

If the SDK path is placed into PG_SYSROOT instead, anyone who builds an extension against the server may point it to a setting that is more fitting to their environment.

This has been reported a few times by users of pgvector, with one of the latest being pgvector/pgvector#592 (comment)

[tureba]

Reported to me by Andrew Kane.

@ankane, does this change look reasonable to you?

[ankane]

Big thanks @tureba! I believe that'll fix it. I think it'd be good to bump SDK_PATH to MacOSX15.sdk as well. (edit: this isn't really needed / can wait)

To share more context, C extensions currently fail to compile unless users have a specific version of the SDK installed (pgvector reports).

It looks like -isysroot is currently included in both --cflags and --cppflags (with --cppflags taking precedence):

/Library/PostgreSQL/17/bin/pg_config --cflags
# ... -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk ...
/Library/PostgreSQL/17/bin/pg_config --cppflags
# ... -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk ...

Compared to Homebrew Postgres, which doesn't have this issue:

/opt/homebrew/opt/postgresql@17/bin/pg_config --cflags  
# no sysroot
/opt/homebrew/opt/postgresql@17/bin/pg_config --cppflags
# ... -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk ...

Setting it to a major version of the SDK like Homebrew does would help mitigate this.

There's also a symlink to the latest installed version at /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk, which would allow extensions to work with any version of the SDK if SDK_PATH was set to that. However, the default Postgres logic seems to prefer a versioned one (relevant commit).

[tureba]

@sandeep-edb , I think you might be the right person to review this.

The change I proposed should cover the initial complaint of users not being able to build extensions against the server.

But Andrew raises some further points about the version of the SDK and differences between cflags and cppflags, which is a subject I know nothing about. Thoughts on them?

[ankane]

Thanks @tureba, the additional context I added was probably more confusing than helpful.

Your PR should both fix the flags and set the SDK/sysroot to a major version.

[sandeep-edb]

Thanks @tureba @ankane this makes sense. Thanks for the PR and the additional context. It fixes both the issues. Will incorporate this change for v18 and the back branches.