Code

Add support for Flash attention#725

Code

Merged

Quentin-Anthony merged 5 commits intoEleutherAI:mainEleutherAI/gpt-neox:mainfrom VHellendoorn:mainVHellendoorn/gpt-neox:mainCopy head branch name to clipboardDec 10, 2022

Commits

Commits on Nov 30, 2022

• Add support for Flash attention

  VHellendoorn

  committed
  6a072ba

  Copy full SHA for 6a072ba

Commits on Dec 6, 2022

• Fix attention type can be both sparse and flash

  VHellendoorn

  authored
  320f5ac

  Copy full SHA for 320f5ac

Commits on Dec 7, 2022

• Updates from running pre-commit on modified files

  VHellendoorn

  committed
  edd1c47

  Copy full SHA for edd1c47

• Merge branch 'main' into main

  StellaAthena

  authored
  f37be02

  Copy full SHA for f37be02

Commits on Dec 10, 2022

• Update README.md

  Quentin-Anthony

  authored
  244334b

  Copy full SHA for 244334b