Add Polly support. by DiamondLovesYou · Pull Request #51061 · rust-lang/rust

DiamondLovesYou · 2018-05-25T16:25:27Z

Use can be triggered via -Z polly, when rustc uses an LLVM which includes polly.

Force LLVM rebuild on buildbots.

rust-highfive · 2018-05-25T16:25:30Z

r? @estebank

(rust_highfive has picked a reviewer for you, use r? to override)

rust-highfive · 2018-05-25T16:25:31Z

Warning

These commits modify submodules.

DiamondLovesYou · 2018-05-25T16:59:54Z

r? @kennytm @ollie27

kennytm · 2018-05-25T17:08:40Z

Previous discussion: #50044.

kennytm · 2018-05-27T07:23:33Z

@bors r+

bors · 2018-05-27T07:23:34Z

📌 Commit 27586fd has been approved by kennytm

kennytm · 2018-05-27T07:24:54Z

cc #39884

bors · 2018-05-27T09:07:52Z

⌛ Testing commit 27586fd2d4e1db8e30caf7423406ca18c50b14e5 with merge 15d620c54f78dcc994cb706abf7be4b2b0c8ba03...

bors · 2018-05-27T09:22:23Z

💔 Test failed - status-appveyor

kennytm · 2018-05-27T10:00:27Z

It caused a bunch on undefined references when linking lib/LLVMPolly.dll (dll?) on Windows (i686-mingw-1).

[00:12:16] [2027/2254] Linking CXX shared module lib\LLVMPolly.dll
[00:12:16] FAILED: lib/LLVMPolly.dll 
[00:12:16] cmd.exe /C "cd . && C:\projects\rust\sccache.exe g++.exe  -ffunction-sections -fdata-sections -m32 -fno-omit-frame-pointer -Werror=date-time -std=gnu++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fno-common -Woverloaded-virtual -Wno-long-long -Wall -W -Wno-unused-parameter -Wwrite-strings -fno-exceptions -fno-rtti  -O2   -shared -o lib\LLVMPolly.dll -Wl,--major-image-version,0,--minor-image-version,0 tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/DependenceInfo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/PolyhedralInfo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopDetection.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopDetectionDiagnostic.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopInfo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopGraphPrinter.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/ScopPass.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/PruneUnprofitable.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/BlockGenerators.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/IslAst.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/IslExprBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/IslNodeBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/CodeGeneration.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/LoopGenerators.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/IRBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/Utils.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/RuntimeDebugBuilder.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/CodegenCleanup.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/CodeGen/PerfMonitor.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Exchange/JSONExporter.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/GICHelper.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/SCEVAffinator.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/SCEVValidator.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/RegisterPasses.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/ScopHelper.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/ScopLocation.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/ISLTools.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/DumpModulePass.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Support/VirtualInstruction.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/External/JSON/json_reader.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/External/JSON/json_value.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/External/JSON/json_writer.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/Canonicalization.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/CodePreparation.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/DeadCodeElimination.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/ScheduleOptimizer.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/FlattenSchedule.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/FlattenAlgo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/ForwardOpTree.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/DeLICM.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/ZoneAlgo.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/Simplify.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/MaximalStaticExpansion.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/RewriteByReferenceParameters.cpp.obj tools/polly/lib/CMakeFiles/PollyCore.dir/Transform/ScopInliner.cpp.obj tools/polly/lib/CMakeFiles/LLVMPolly.dir/Polly.cpp.obj  lib/libPollyISL.a -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 && cd ."
[00:12:16] tools/polly/lib/CMakeFiles/PollyCore.dir/Analysis/DependenceInfo.cpp.obj:DependenceInfo.cpp:(.text$__tcf_2+0x11): undefined reference to `vtable for llvm::cl::Option'
...
[00:12:16] tools/polly/lib/CMakeFiles/LLVMPolly.dir/Polly.cpp.obj:Polly.cpp:(.text.startup._GLOBAL__sub_I_Polly.cpp+0x7): undefined reference to `llvm::PassRegistry::getPassRegistry()'
[00:12:16] collect2.exe: error: ld returned 1 exit status

DiamondLovesYou · 2018-05-27T21:06:38Z

Huh, that's odd. Rust shouldn't ever try to link polly dynamically, due to the fact that it's in LLVM*.* form and not libLLVM*.* form. So we could, as a last resort, just not build Polly when the host is Windows. I don't develop on windows normally (or at all really); let me setup my gaming windows VM to try and reproduce this. It would be unfortunate to have to disable this for Windows devs.

bors · 2018-05-30T16:40:51Z

☔ The latest upstream changes (presumably #50955) made this pull request unmergeable. Please resolve the merge conflicts.

mati865 · 2018-09-06T22:19:33Z

At MSYS2 we have disabled LLVMPolly because of similar error, it is proven to build but wasn't tested much. Here is the patch: https://github.com/Alexpux/MINGW-packages/blob/5cbc0add24760165f557fd00bd9d915279ea4611/mingw-w64-clang/0801-Don-t-build-LLVMPolly-on-WIN32.patch

bors · 2018-09-07T15:24:06Z

☔ The latest upstream changes (presumably #54021) made this pull request unmergeable. Please resolve the merge conflicts.

DiamondLovesYou · 2018-09-10T22:25:08Z

K, for future reference: on Msys2, use mingw-w64-x86_64-python2, not python2. Finally got LLVM to configure correctly.

DiamondLovesYou · 2018-09-11T01:45:08Z

Patch to make Polly not try to build the loadable modules on all Windows platforms: https://reviews.llvm.org/D51904.

@mati865 I made a minor change: I used LLVM_ENABLE_PLUGINS instead of checking platform specific booleans.

mati865 · 2018-09-11T10:58:13Z

@DiamondLovesYou MSYS2 builds LLVM with LLVM_BUILD_LLVM_DYLIB so it has LLVM_ENABLE_PLUGINS enabled and Polly doesn't build.

As long as Rust doesn't enable BUILD_SHARED_LIBS or LLVM_BUILD_LLVM_DYLIB for LLVM Polly should build on Windows.

DiamondLovesYou · 2018-09-11T23:48:20Z

@mati865 Okay, I dug in and managed to (just now) fix the original issue. I built LLVM with LLVM_BUILD_LLVM_DYLIB && LLVM_LINK_LLVM_DYLIB on Msys/mingw/whatnot without disabling LLVMPolly!

Sadly, using LINK_POLLY_INTO_TOOLS and LLVM_LINK_LLVM_DYLIB together breaks some tools as Polly is linked into the dylib already.

mati865 · 2018-09-12T00:06:11Z

@DiamondLovesYou nice work but maybe you are going too deep? This is pretty much uncharted territory.
Ofc having it fully working everwhere would be nice but it's a lot of effort and time.
This PR seems to be already on pair with LLVM with Polly working on UNIX like platforms and not regress on Windows.

DiamondLovesYou · 2018-09-12T01:51:07Z

@mati865 Maybe, but it doesn't matter now.

New patch: https://reviews.llvm.org/D51963.

bors · 2018-09-12T11:38:33Z

☔ The latest upstream changes (presumably #54146) made this pull request unmergeable. Please resolve the merge conflicts.

DiamondLovesYou · 2018-09-12T14:48:10Z

@mati865 https://reviews.llvm.org/D51984 and https://reviews.llvm.org/D51986

lovesegfault · 2018-09-14T06:50:40Z

@DiamondLovesYou You're an absolute hero for pushing this through, thank you so much! Hope to see those Polly patches landing soon.

DiamondLovesYou · 2018-09-17T16:11:04Z

Thanks! Hopefully I can actually get it in; a lot of platform specific issues are coming up, which are breaking check-all in different ways for the BUILD_SHARED_LIBS && LLVM_LINK_LLVM_DYLIB && platform combinations.

…uses an LLVM which includes polly. Force LLVM rebuild on buildbots.

matthiaskrgr · 2018-10-19T00:49:44Z

.gitmodules

-
+[submodule "src/polly"]
+	path = src/polly
+	url = https://github.com/llvm-mirror/polly.git


Perhaps rust-lang should have its own fork of this repo?
I could imagine that if upstream polly gets synced with upstream llvm but rustc uses rust-lang-llvm with llvm upstream polly, there might be conflicts between let's say polly 8.0 and llvm 7.0 (the rustc-lang fork).

We could track the release_70 branch instead of master if version incompatibility is the only concern.

bors · 2018-10-21T01:13:47Z

☔ The latest upstream changes (presumably #55230) made this pull request unmergeable. Please resolve the merge conflicts.

Centril · 2018-12-01T13:44:51Z

Ping from triage, @kennytm: should this PR be closed? @DiamondLovesYou any updates?

kennytm · 2018-12-01T13:52:23Z

@Centril this was blocked by several patched on LLVM. If OP did not have any updates this could be closed as S-blocked-closed.

Centril · 2018-12-01T13:55:41Z

Alright; closing as S-blocked-closed for the time being then.

rust-highfive assigned estebank May 25, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 25, 2018

DiamondLovesYou force-pushed the polly branch from e67fd06 to 27586fd Compare May 25, 2018 16:55

rust-highfive assigned kennytm and unassigned estebank May 25, 2018

kennytm approved these changes May 27, 2018

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 27, 2018

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 27, 2018

kennytm added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 27, 2018

DiamondLovesYou force-pushed the polly branch from 27586fd to 1eb602e Compare June 6, 2018 00:19

kennytm added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 7, 2018

DiamondLovesYou force-pushed the polly branch 2 times, most recently from e208473 to 6e97e37 Compare September 10, 2018 16:06

DiamondLovesYou force-pushed the polly branch from 6e97e37 to 0225bbd Compare September 10, 2018 23:36

kennytm added S-blocked Status: Blocked on something else such as an RFC or other implementation work. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 11, 2018

Add Polly support. Use can be triggered via -Z polly, when rustc …

d51fa91

…uses an LLVM which includes polly. Force LLVM rebuild on buildbots.

DiamondLovesYou force-pushed the polly branch from 0225bbd to d51fa91 Compare October 18, 2018 23:38

matthiaskrgr reviewed Oct 19, 2018

View reviewed changes

steveklabnik mentioned this pull request Nov 20, 2018

Provide an easy way to use LLVM's Polly #39884

Open

5 tasks

Centril removed the S-blocked Status: Blocked on something else such as an RFC or other implementation work. label Dec 1, 2018

namibj mentioned this pull request Aug 17, 2020

Add Polly Support #75615

Closed

JRF63 mentioned this pull request Oct 30, 2020

Enable LLVM Polly via llvm-args. #78566

Merged

Uh oh!

Conversation

DiamondLovesYou commented May 25, 2018

Uh oh!

rust-highfive commented May 25, 2018

Uh oh!

rust-highfive commented May 25, 2018

Uh oh!

DiamondLovesYou commented May 25, 2018

Uh oh!

kennytm commented May 25, 2018

Uh oh!

kennytm commented May 27, 2018

Uh oh!

bors commented May 27, 2018

Uh oh!

kennytm commented May 27, 2018

Uh oh!

bors commented May 27, 2018

Uh oh!

bors commented May 27, 2018

Uh oh!

kennytm commented May 27, 2018

Uh oh!

DiamondLovesYou commented May 27, 2018

Uh oh!

bors commented May 30, 2018

Uh oh!

mati865 commented Sep 6, 2018

Uh oh!

bors commented Sep 7, 2018

Uh oh!

DiamondLovesYou commented Sep 10, 2018

Uh oh!

DiamondLovesYou commented Sep 11, 2018

Uh oh!

mati865 commented Sep 11, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DiamondLovesYou commented Sep 11, 2018

Uh oh!

mati865 commented Sep 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DiamondLovesYou commented Sep 12, 2018

Uh oh!

bors commented Sep 12, 2018

Uh oh!

DiamondLovesYou commented Sep 12, 2018

Uh oh!

lovesegfault commented Sep 14, 2018

Uh oh!

DiamondLovesYou commented Sep 17, 2018

Uh oh!

matthiaskrgr Oct 19, 2018

Choose a reason for hiding this comment

Uh oh!

kennytm Oct 19, 2018

Choose a reason for hiding this comment

Uh oh!

bors commented Oct 21, 2018

Uh oh!

Centril commented Dec 1, 2018

Uh oh!

kennytm commented Dec 1, 2018

Uh oh!

Centril commented Dec 1, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

mati865 commented Sep 11, 2018 •

edited

Loading

mati865 commented Sep 12, 2018 •

edited

Loading