Understanding GPT - Transformers
AI
Part 2/3 - Understanding how modern LLMS work. From RNNs, to transformers, towards modern scaling laws.
AI
Top
Adventures in ML trading - Part 1
ML Trading
Part 1/3 - Exporing the mathematical, statistical, and probabilistic nature of the market. Specifically, I attempt building a mean-reversion probability model, backtesting it against historical data, and understanding where/why it falls short. The results explain why simple statistical models fail to capture the complex beast that is the financial market. Nevertheless, this helps with foundational understanding and there is much to learn that I then iterate in the subsequent posts on the topic of ML based Trading.
Exploring Code LLMs - Instruction fine-tuning, models and quantization
AI
Part 1/3 - Evaluating LLM’s that are specialised in code generation tasks, and evaluating their performance on writing code. This post starts with concepts and theory, while the next 2 parts evaluate specific code models.
Getting Things Done with LogSeq
Management
Introduction I was first introduced to the concept of “second-brain” from Tobi Lutke, the founder of Shopify. The topic started because someone asked whether he still codes - now that he is a founder of such a large company. Tobi went on to explain that he spent the weekend writing some code to customise Logseq to his preferences, and that he’s an active member of the Logseq community.
The following weekend, I setup Logseq and learnt its weird ways of working, and have since been an ardent user and fan of the Logseq/Obsidian methodology of building a “second-brain”...
All posts
Understanding GPT - A Journey from RNNs to Attention
Machine Learning
Introduction ChatGPT has took the world by storm, and has possibly started the 6th wave. Given its importance, the rush to build new products and research on top is understandable. But, I’ve always liked to ground myself with foundational knowledge on how things work, before exploring anything additive. To gain such foundational knowledge, I believe understanding the progression of techniques and models is important to comprehend and appreciate how these LLM models work under the hood....
Loss Functions in ML
Machine Learning
Introduction Loss functions tell the algorithm how far we are from actual truth, and their gradients/derivates help understand how to reduce the overall loss (by changing the parameters being trained on) All losses in keras defined here But why is the loss function expressed as a negative loss? Plot: As probabilities only lie between [0-1], the plot is only relevant between X from 0-1 This means, that it penalises a low probability of success exponentially more....