Thonk From First Principles
Horace HeML Systems from first principles. Aims to be better than a ChatGPT summary.
| Platform |  | Pricing | Freemium | Publishes | Monthly |
|---|---|---|---|---|---|
| Issues | 6 | Founded | 3 years ago | Last Issue | a year ago |
| Active | |||||
Read this Newsletter
thonking.aiLatest Issues
Why PyTorch is an amazing place to work... and Why I'm Joining Thinking Machines
Note: This is a revised (and extended) version of the note I posted internally when I decided to leave PyTorch. I’ve gotten a lot of questions, and I hope that this post will both 1. answer those questions, 2. convince you that working at P...
FlexAttention: The Flexibility of PyTorch with the Performance of FlashAttention [external]
I have a blog post up on the PyTorch blog (as part of my day job) on FlexAttention: https://pytorch.org/blog/flexattention/ (work done with Driss Guessous, Yanbo Liang, and Joy Dong). And here’s a tweet thread.
The beginning excerpted here...
Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short]
It’s 2022. I check out this cool new project, CUTLASS, with very fast matmuls. I take a large matmul, 8192 x 8192 x 8192, and benchmark it in PyTorch, which calls CuBLAS.
python mm_bench.py
> CuBLAS: 258 Teraflops
Not bad...
Solutions: What Shapes Do Matrix Multiplications Like?
Companion to https://www.thonking.ai/p/what-shapes-do-matrix-multiplications
Note: The answer to question 1 is publicly available, but the answers to the rest are paywalled. However, if you write up your solutions to eaach question and mes...
What Shapes Do Matrix Multiplications Like? [medium]
A while back, Karpathy tweeted that increasing the size of his matmul made it run faster. Surprisingly, it’s not just relatively faster, it takes less absolute time. In other words, despite doing more work, it is executing in less time.
ht...
Audience Metrics
Subscribers, engagement, traffic and sponsorship for Thonk From First Principles.
| Subscribers | Monthly Web Visits | Accepts Sponsors | |||
|---|---|---|---|---|---|
| Estimated Cost per Ad | |||||
Explore More
SEO Reach
Where Thonk From First Principles ranks on Google, and how much search traffic it brings in.
| Ranked Keywords | 33 | Monthly Search Traffic | Top Keywords |
|---|
Authors
The writers behind this newsletter.
Frequently Asked Questions
How can I access the email archive for Thonk From First Principles?
You can find recent issues that have been published by Thonk From First Principles on Reletter by scrolling up to where it says Latest Issues. Tap on the link for any of the most recent emails or hit More Issues to see older ones.
How many subscribers does Thonk From First Principles have?
To see how many people subscribe to Thonk From First Principles, simply upgrade your Reletter account. We provide readership numbers and lots of other stats for this newsletter so you can decide if it's worth reaching out to.
How can I advertise in Thonk From First Principles?
Newsletter advertising can be extremely effective when it's done right. Before you pitch Thonk From First Principles as a potential sponsor or partner, make sure that you've done your research and checked its newsletter stats with Reletter.
Then, personalize one of our winning pitching templates and send it to the right person using the contact info provided.
How much does it cost to sponsor a publication like Thonk From First Principles?
Newsletter ad rates (or CPM) vary depending on many factors, including industry, number of subscribers, open rate, ad placement and more.
To find out how much an ad will cost, contact Thonk From First Principles using the contact information provided and ask for a copy of their media kit.
How can I find newsletters related to Thonk From First Principles?
Scroll up to where it says Related Newsletters to see other publications like Thonk From First Principles. You can also search our email newsletter directory to discover other newsletters that cover the topics you're interested in.
How do I contact Thonk From First Principles?
Reletter provides this newsletter's website URL above, where you will often find their contact information. We also provide links to associated social media accounts and pitching templates so you can reach out fast.