[Feature Request]: Support f16 via the half crate

[akern40]

Hello! We are considering support for f16 (and bf16) via the half crate in ndarray (rust-ndarray/ndarray#1551), but we are seeing rather dismal performance on matrix multiplication for the new types: f16 appears to be ~3 orders of magnitude slower than f32. After some debugging, I believe this is a testament to matrixmultiply's performance: the code on my Apple M2 chip is hitting the f16 assembly instructions, so I think most of the performance difference is thanks to matrixmultiply's very fast sgemm.

In light of this, I was wondering what the appetite would be for supporting f16 here in the matrixmultiply crate.

cc: @swfsql, who has been the champion for f16 in ndarray.

[bluss]

At this time, please look at crate gemm-f16 first.

To be interesting here we'd want it to be viable - which arches have SIMD support in stable Rust? - and for which kinds (both f16 and bf16)? And we want it to be useful as part of this crate.

[swfsql]

Thank you both for considering this, I'll try to respond to the questions. Please note I'm quite ignorant on SIMD in overall.

I believe that types from half (which brings f16/bf16 to stable, even if they are currently slow) are mostly related to the machine learning ecosystem (specially for debugging, or possibly for edge devices). Also, currently no stable arch has stable SIMD support for f16/bf16. Bf16 (compared against f16) is not considered a priority in overall as far as SIMD stabilization go (I also don't think there are too many CPU intrinsics that targeted that).

So I think the ecosystem in overall is still "getting there" for f16 SIMD support, and it might later evolve to bf16.

For what is worth, I read that the half types are layout compatible to the unstable f16 types, so SIMD advancements to native types are somewhat expected to benefit the half types as well.

[bluss]

from gemm-f16 crate it looks like aarch64 (neon) might have stable support for float16 intrinsics, that's cool. I'm sorry but I can not look at this right now, try to use the existing crate.