λx:Math.Metal

from calculus to compilers

calculus

compilers

static: theano, torch7, tensorflow

torch: https://publications.idiap.ch/downloads/reports/2002/rr02-46.pdf

dyanmic, non-dsl: autograd, chainer, pytorch

autograd (hips. numpy. "define-by-run"): https://indico.ijclab.in2p3.fr/event/2914/contributions/6483/subcontributions/180/attachments/6060/7185/automl-short.pdf chainer ("define by run" for gpus): https://arxiv.org/pdf/1908.00213 tensorflow introduced "eager mode"

jax: speed.

mojo: perf pitch. cross compile pitch.

Rally points to get rid of CUDA's moat (swamp)

backends

MLIR

XLA

Triton

https://x.com/soumithchintala/status/1736555740448362890 https://x.com/marksaroufim/status/1722386684975984950 https://x.com/marksaroufim/status/1600917846954893314 https://www.marksaroufim.com/2022/08/31/tour-of-torchdata.html https://arxiv.org/abs/2002.03794 https://arxiv.org/pdf/2312.15234 https://scholar.google.com/citations?user=7nlvOMQAAAAJ&hl=en http://blog.ezyang.com/2019/05/pytorch-internals/ https://apaszke.github.io/torch-internals.html https://scholar.google.com.au/citations?view_op=view_citation&hl=en&user=36ofBJgAAAAJ&citation_for_view=36ofBJgAAAAJ:4JMBOYKVnBMC https://scholar.google.com.au/citations?view_op=view_citation&hl=en&user=36ofBJgAAAAJ&citation_for_view=36ofBJgAAAAJ:ZeXyd9-uunAC https://scholar.google.com.au/citations?view_op=view_citation&hl=en&user=36ofBJgAAAAJ&citation_for_view=36ofBJgAAAAJ:TFP_iSt0sucC https://scholar.google.com.au/citations?view_op=view_citation&hl=en&user=36ofBJgAAAAJ&citation_for_view=36ofBJgAAAAJ:O3NaXMp0MMsC https://scholar.google.com.au/citations?view_op=view_citation&hl=en&user=36ofBJgAAAAJ&cstart=20&pagesize=80&citation_for_view=36ofBJgAAAAJ:GnPB-g6toBAC https://scholar.google.com.au/citations?view_op=view_citation&hl=en&user=36ofBJgAAAAJ&cstart=20&pagesize=80&citation_for_view=36ofBJgAAAAJ:nb7KW1ujOQ8C https://hy3na.com/posts/parallelism.html https://hy3na.com/posts/compilers.html https://hy3na.com/posts/polyhedral.html https://x.com/marksaroufim/status/1826379347915407494 jit: https://x.com/ezyang/status/1513889371463766020 https://kuterdinel.com/writing-a-very-simple-jit-compiler-in-about-1000-lines-of-c.html