Writing Speed-of-Light Flash Attention for 5090 in CUDA C++

(gau-nernst.github.io)

81 points | by dsr12 5 hours ago

3 comments

  • steinvakt2 1 hour ago
    I had a 5090 some months ago but couldnt get flash attention to work. Does it now work natively? What about 5080?
  • ProofHouse 1 hour ago
    Damn awesome. This going to take me 3 reads and a week to digest
  • doctorpangloss 1 hour ago
    Hmm, but supposing the accelerated NVIDIA specific inference data types were available for Triton, then you would just use that? Why not contribute to Triton, they accept PRs? Like so what if you do free product ecosystem development for NVIDIA and giant corporations by contributing to Triton?
    • qeternity 1 hour ago
      Second line of the post:

      > The main objective is to learn writing attention in CUDA C++, since many features are not available in Triton, such as MXFP8 / NVFP4 MMA for sm120.