Skip to content

Add support for Mac OS#18

Draft
aszlig wants to merge 4 commits into
mainfrom
darwin
Draft

Add support for Mac OS#18
aszlig wants to merge 4 commits into
mainfrom
darwin

Conversation

@aszlig

@aszlig aszlig commented Jul 8, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor

This is a WIP branch to implement support for Mac OS X, which in theory should work since it has a similar mechanism (DYLD_INSERT_LIBRARIES) to LD_PRELOAD on GNU/Linux.

Fundamental issues:

  • Find a way to handle PID/GID in LOCAL_PEERCRED. It seems that there is only xucred.cr_uid, but in order to provide fake client IPs we need at least the peer PID to properly distinguish the individual peers.
  • We can't use std::filesystem yet, because Ubuntu 18.04 is still using GCC 7. Find a less ugly way to get_current_dir_name in a cross-platform way. According to https://ubuntu.com/18-04, the standard support has expired on 31 May 2023.

@aszlig aszlig force-pushed the darwin branch 5 times, most recently from 5887a4b to 5911036 Compare July 9, 2020 06:55
@aszlig aszlig added the help wanted Extra attention is needed label Jul 10, 2020
@aszlig aszlig mentioned this pull request Oct 20, 2020
Closed
@aszlig aszlig changed the title Add support for Mac OS X Add support for Mac OS Jul 8, 2021
@aszlig aszlig force-pushed the darwin branch 2 times, most recently from a8c872a to 3ef2ee3 Compare July 8, 2021 22:55
@tiboratAS tiboratAS mentioned this pull request Dec 3, 2021
Closed
@aszlig

aszlig commented Aug 6, 2023

Copy link
Copy Markdown
Contributor Author

For the record and according to golang/go#27613, there seems to be a LOCAL_PEERPID socket option that we could possibly use.

aszlig added 2 commits August 6, 2023 17:56
@aszlig
Revert "github: Remove workflow for Mac OS X"
3790608 
This reverts commit c945f58.

I initially removed the workflow because it was failing all the time and
always seeing commit messages with failed CI checks is not only pretty
unhelpful but we might miss other *relevant* failures because of that.

So this brings back the workflow for Darwin since we're now actually
working on at least trying to implement support for it.

The reason why I'm not adding this *after* adding support is that I
don't have a Mac OS X machine, so I have to purely relying on the GitHub
Actions workflow.

Signed-off-by: aszlig <aszlig@nix.build>
@aszlig
darwin: Use LOCAL_PEERCRED instead of SO_PEERCRED
3581ecb 
While LOCAL_PEERCRED is somewhat similar to SO_PEERCRED, we
unfortunately don't have access to the PID of the remote peer.

This is something we actually need to properly distinguish the remote
peer by giving it an IP address with the PID encoded, otherwise we'd end
up with duplicate IPs.

On the other hand, using random IP addresses also is not a very good
solution here, since we actually *want* to have the same IP for the same
process.

Right now the UID and GID fields are not used at all on Darwin, but we
really need to figure out a way to properly assign fake IP addresses.

Signed-off-by: aszlig <aszlig@nix.build>
aszlig added 2 commits August 6, 2023 18:03
@aszlig
github: Add pytest to macos workflow
714d2a8 
This is needed in order to run integration tests, which we certainly
want to run on Darwin, because I do not have a single machine running
MacOS and thus for me the only way to check whether something is broken
is by checking whether the integration tests have failed.

Signed-off-by: aszlig <aszlig@nix.build>
@aszlig
fixup! Revert "github: Remove workflow for Mac OS X"
a4efd83
aszlig added a commit that referenced this pull request May 20, 2026
@aszlig
Only access non-locals after they're initialised
f131afc 
This fixes a long-standing issue on aarch64, which is caused by mimalloc
in Python 3 calling close() too early:

  #0  0x0000007ff75952ac in __pthread_kill_implementation () from .../lib/libc.so.6
  #1  0x0000007ff753c5b8 in raise () from .../lib/libc.so.6
  #2  0x0000007ff7525d38 in abort () from .../lib/libc.so.6
  #3  0x0000007ff7299818 in __gnu_cxx::__verbose_terminate_handler() () from .../lib/libstdc++.so.6
  #4  0x0000007ff729704c in __cxxabiv1::__terminate(void (*)()) () from .../lib/libstdc++.so.6
  #5  0x0000007ff728e49c in std::terminate() () from .../lib/libstdc++.so.6
  #6  0x0000007ff72973dc in __cxa_throw () from .../lib/libstdc++.so.6
  #7  0x0000007ff728ed7c in std::__throw_bad_array_new_length() () from .../lib/libstdc++.so.6
  #8  0x0000007ff7f10c54 in std::__new_allocator<std::__detail::_Hash_node_base*>::allocate (this=0x7fffffe890, __n=18446744073709551557) at .../include/c++/15.2.0/bits/new_allocator.h:139
  #9  0x0000007ff7f6e898 in std::allocator_traits<std::allocator<std::__detail::_Hash_node_base*> >::allocate (__a=..., __n=18446744073709551557)
      at .../include/c++/15.2.0/bits/alloc_traits.h:614
  #10 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<int, false> > >::_M_allocate_buckets (this=0x7ff7fb02a0 <all_fds>, __bkt_count=18446744073709551557)
      at .../include/c++/15.2.0/bits/hashtable_policy.h:1605
  #11 0x0000007ff7f6e500 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::_M_allocate_buckets (this=0x7ff7fb02a0 <all_fds>, __bkt_count=18446744073709551557)
      at .../include/c++/15.2.0/bits/hashtable.h:413
  #12 0x0000007ff7f6d944 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::_M_rehash (this=0x7ff7fb02a0 <all_fds>, __bkt_count=18446744073709551557) at .../include/c++/15.2.0/bits/hashtable.h:2754
  #13 0x0000007ff7f6c0b8 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::_M_insert_unique_node (this=0x7ff7fb02a0 <all_fds>, __bkt=3, __code=3, __node=0x5555583580, __n_elt=1)
      at .../include/c++/15.2.0/bits/hashtable.h:2479
  #14 0x0000007ff7f6a034 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::_M_emplace_uniq<int const&> (this=0x7ff7fb02a0 <all_fds>) at .../include/c++/15.2.0/bits/hashtable.h:2365
  #15 0x0000007ff7f685a8 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::insert (this=0x7ff7fb02a0 <all_fds>, __v=@0x7fffffebb8: 3) at .../include/c++/15.2.0/bits/hashtable.h:1049
  #16 0x0000007ff7f66b60 in std::unordered_set<int, std::hash<int>, std::equal_to<int>, std::allocator<int> >::insert (this=0x7ff7fb02a0 <all_fds>, __x=@0x7fffffebb8: 3)
      at .../include/c++/15.2.0/bits/unordered_set.h:475
  #17 0x0000007ff7f64c80 in Systemd::init (rules=...) at ../src/systemd.cc:213
  #18 0x0000007ff7f14bd0 in init_rules () at ../src/preload.cc:77
  #19 0x0000007ff7f19edc in ip2unix_wrap_close (fd=4) at ../src/preload.cc:677
  #20 0x0000007ff78cf430 in mi_process_init.part () from .../lib/libpython3.13.so.1.0
  #21 0x0000007ff7894ee8 in _mi_process_init () from .../lib/libpython3.13.so.1.0
  #22 0x0000007ff7fc32c4 in call_init () from .../lib/ld-linux-aarch64.so.1
  #23 0x0000007ff7fc341c in _dl_init () from .../lib/ld-linux-aarch64.so.1
  #24 0x0000007ff7fdbea0 in _start () from .../lib/ld-linux-aarch64.so.1

Here the non-local all_fds variable is an unordered_set which hasn't yet
been initialised but is accessed because _mi_process_init has a lower
priority and thus is initialised *before* all_fds.

Since mi_process_init calls close(), an insert on the all_fds variable
is triggered by the systemd rule initialisation, which then fails
because all_fds hasn't been initialised yet.

The all_fds variable isn't the only one that's affected, so in order to
avoid wrapping all non-local variables in lazy initalisation wrappers,
we now short-circuit all the library calls that trigger the ruleset
initialisation machinery.

While at it, I also fixed a potential data race that could emerge in
multithreaded programs. While in practice most networking code is
initialised before actually spawning threads, it's still something that
could happen, so let's use std::call_once on rule initialisation.

Signed-off-by: aszlig <aszlig@nix.build>
aszlig added a commit that referenced this pull request May 20, 2026
@aszlig
Only access non-locals after they're initialised
e8ebfe6 
This fixes a long-standing issue on aarch64, which is caused by mimalloc
in Python 3 calling close() too early:

  #0  0x0000007ff75952ac in __pthread_kill_implementation () from .../lib/libc.so.6
  #1  0x0000007ff753c5b8 in raise () from .../lib/libc.so.6
  #2  0x0000007ff7525d38 in abort () from .../lib/libc.so.6
  #3  0x0000007ff7299818 in __gnu_cxx::__verbose_terminate_handler() () from .../lib/libstdc++.so.6
  #4  0x0000007ff729704c in __cxxabiv1::__terminate(void (*)()) () from .../lib/libstdc++.so.6
  #5  0x0000007ff728e49c in std::terminate() () from .../lib/libstdc++.so.6
  #6  0x0000007ff72973dc in __cxa_throw () from .../lib/libstdc++.so.6
  #7  0x0000007ff728ed7c in std::__throw_bad_array_new_length() () from .../lib/libstdc++.so.6
  #8  0x0000007ff7f10c54 in std::__new_allocator<std::__detail::_Hash_node_base*>::allocate (this=0x7fffffe890, __n=18446744073709551557) at .../include/c++/15.2.0/bits/new_allocator.h:139
  #9  0x0000007ff7f6e898 in std::allocator_traits<std::allocator<std::__detail::_Hash_node_base*> >::allocate (__a=..., __n=18446744073709551557)
      at .../include/c++/15.2.0/bits/alloc_traits.h:614
  #10 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<int, false> > >::_M_allocate_buckets (this=0x7ff7fb02a0 <all_fds>, __bkt_count=18446744073709551557)
      at .../include/c++/15.2.0/bits/hashtable_policy.h:1605
  #11 0x0000007ff7f6e500 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::_M_allocate_buckets (this=0x7ff7fb02a0 <all_fds>, __bkt_count=18446744073709551557)
      at .../include/c++/15.2.0/bits/hashtable.h:413
  #12 0x0000007ff7f6d944 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::_M_rehash (this=0x7ff7fb02a0 <all_fds>, __bkt_count=18446744073709551557) at .../include/c++/15.2.0/bits/hashtable.h:2754
  #13 0x0000007ff7f6c0b8 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::_M_insert_unique_node (this=0x7ff7fb02a0 <all_fds>, __bkt=3, __code=3, __node=0x5555583580, __n_elt=1)
      at .../include/c++/15.2.0/bits/hashtable.h:2479
  #14 0x0000007ff7f6a034 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::_M_emplace_uniq<int const&> (this=0x7ff7fb02a0 <all_fds>) at .../include/c++/15.2.0/bits/hashtable.h:2365
  #15 0x0000007ff7f685a8 in std::_Hashtable<int, int, std::allocator<int>, std::__detail::_Identity, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::insert (this=0x7ff7fb02a0 <all_fds>, __v=@0x7fffffebb8: 3) at .../include/c++/15.2.0/bits/hashtable.h:1049
  #16 0x0000007ff7f66b60 in std::unordered_set<int, std::hash<int>, std::equal_to<int>, std::allocator<int> >::insert (this=0x7ff7fb02a0 <all_fds>, __x=@0x7fffffebb8: 3)
      at .../include/c++/15.2.0/bits/unordered_set.h:475
  #17 0x0000007ff7f64c80 in Systemd::init (rules=...) at ../src/systemd.cc:213
  #18 0x0000007ff7f14bd0 in init_rules () at ../src/preload.cc:77
  #19 0x0000007ff7f19edc in ip2unix_wrap_close (fd=4) at ../src/preload.cc:677
  #20 0x0000007ff78cf430 in mi_process_init.part () from .../lib/libpython3.13.so.1.0
  #21 0x0000007ff7894ee8 in _mi_process_init () from .../lib/libpython3.13.so.1.0
  #22 0x0000007ff7fc32c4 in call_init () from .../lib/ld-linux-aarch64.so.1
  #23 0x0000007ff7fc341c in _dl_init () from .../lib/ld-linux-aarch64.so.1
  #24 0x0000007ff7fdbea0 in _start () from .../lib/ld-linux-aarch64.so.1

Here the non-local all_fds variable is an unordered_set which hasn't yet
been initialised but is accessed because _mi_process_init has a lower
priority and thus is initialised *before* all_fds.

Since mi_process_init calls close(), an insert on the all_fds variable
is triggered by the systemd rule initialisation, which then fails
because all_fds hasn't been initialised yet.

The all_fds variable isn't the only one that's affected, so in order to
avoid wrapping all non-local variables in lazy initalisation wrappers,
we now short-circuit all the library calls that trigger the ruleset
initialisation machinery.

While at it, I also fixed a potential data race that could emerge in
multithreaded programs. While in practice most networking code is
initialised before actually spawning threads, it's still something that
could happen, so let's use std::call_once on rule initialisation.

Signed-off-by: aszlig <aszlig@nix.build>
Tested-by: Adam Mizerski <adam@mizerski.pl>
Closes: #40
Fixes: #37
ara4711 added a commit to ara4711/ip2unix that referenced this pull request Jun 12, 2026
@ara4711
Make non-preload code portable beyond Linux/glibc
95bfe09 
Preparation for platforms other than GNU/Linux, keeping behaviour on
Linux unchanged:

* Replace SO_PEERCRED's ucred in the SockAddr interface with a
  portable PeerCred struct filled at the call sites; on Darwin it is
  populated from LOCAL_PEERCRED plus LOCAL_PEERPID, since xucred
  carries no pid (the open question in issue nixcloud#18).

* Gate accept4/dup3 (and their wrappers) on __linux__ and fall back
  to accept/dup2 elsewhere; the fallbacks are equivalent because the
  flags-taking wrappers don't exist on those platforms, so flags is
  always 0.

* Gate F_GETSIG/F_SETSIG and F_GETOWN_EX/F_SETOWN_EX behind feature
  checks with an F_GETOWN/F_SETOWN fallback, and asm/sockios.h behind
  __linux__ (sys/ioctl.h provides SIOCSPGRP/FIOASYNC elsewhere).

* Use execvp instead of the GNU-only execvpe; they are equivalent
  here since we pass the inherited environment anyway.

* Guard all remaining wrapper functions with g_initialised so calls
  arriving before our constructors ran (e.g. from the dynamic
  loader's own bootstrap or other libraries' constructors) pass
  straight through to libc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

help wanted Extra attention is needed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@aszlig