Getting Things Done with LogSeq
Book Review

Introduction I was first introduced to the concept of “second-brain” from Tobi Lutke, the founder of Shopify. The topic started because someone asked whether he still codes - now that he is a founder of such a large company. Tobi went on to explain that he spent the weekend writing some code to customise Logseq to his preferences, and that he’s an active member of the Logseq community. The following weekend, I setup Logseq and learnt its weird ways of working, and have since been an ardent user and fan of the Logseq/Obsidian methodology of building a “second-brain”...

February 16, 2024

Understanding GPT 1, 2 and 3
Machine Learning

Introduction The goal of this series of posts, is to form foundational knowledge that helps us understanding modern state-of-the-art LLM models, and gain a comprehensive understanding of GPT via reading the seminal papers themselves. In my previous post, I covered transformers via the original paper “Attention is all you need” that brought the innovation that made all this progress possible. This post will focus on GPT-3 and its predecessors GPT-1 and 2....

October 1, 2023

Understanding GPT - Transformers
Machine Learning

Introduction The goal of this series of posts, is to form foundational knowledge that helps us understanding modern state-of-the-art LLM models, and gain a comprehensive understanding of GPT via reading the seminal papers themselves. In my previous post, I covered some of the seminal papers that formulated sequence based models from RNNs to the Attention mechanism in encoder-decoder architectures. If you don’t know about them, or would like a quick refresher - I recommend reading through the previous post before continuing here....

July 7, 2023

Understanding GPT - A Journey from RNNs to Attention
Machine Learning

Introduction ChatGPT has rightly taken the world by storm, and has possibly started the 6th wave. Given its importance, the rush to build new products and research on top is understandable. But, I’ve always liked to ground myself with foundational knowledge on how things work, before exploring anything additive. To gain such foundational knowledge, I believe understanding the progression of techniques and models is crucial to comprehend how these LLM models work under the hood....

June 18, 2023

Loss Functions in ML
Machine Learning

Introduction Loss functions tell the algorithm how far we are from actual truth, and their gradients/derivates help understand how to reduce the overall loss (by changing the parameters being trained on) All losses in keras defined here But why is the loss function expressed as a negative loss? Plot: As probabilities only lie between [0-1], the plot is only relevant between X from 0-1 This means, that it penalises a low probability of success exponentially more....

February 18, 2023

Python Cheet Sheet
Cheat Sheets

A quick cheatsheet on python operations Slice: astring = "Hello World" print(astring[3:7]) # prints-> lo w print(astring[0:10:2]) # skips one character, prints -> Hlowr print(astring[::-1]) # reverse a string using step -1 Case astring.upper() astring.lower() Slicing complete list performs a copy spam_copy = spam[:] Zip to loop furniture = ['table', 'chair', 'rack', 'shelf'] price = [100, 50, 80, 40] for item, amount in zip(furniture, price): print(f'The {item} costs ${amount}') Multiple assignments...

January 2, 2023

Grit - Angela Duckworth
Book Summary

Book Summary - Grit by Angela Duckworth The post is a book summary of the main bullet points from the book “Grit” by “Angela Duckworth” Components of Grit Angela breaks down grit in the following components:- Interest: I love what I do Practice: I will do what it takes to improve and become world-class Purpose: What I do is important for everyone Hope: I will keep going even when it’s difficult...

December 12, 2022

Largest Area Under a Histogram (and related concepts/problems).
Algorithms

Problem Statement: GfG quoted: Find the largest rectangular area possible in a given histogram where the largest rectangle can be made of a number of contiguous bars. For simplicity, assume that all bars have same width and the width is 1 unit. The Clever Solution Sometimes, the nicest solutions come from clues we receive from the worst ones. What’s the naive solution ? Iterate through all possible rectangles and calculate the area....

January 30, 2017 · Kshitij Banerjee

Reverse Engineer data from raw database files.

How to recover data from raw .tokudb files. Why? Recently my tokudb database went corrupt after a bad shutdown and a lot of data was now lost. After a lot of googling, asking on forums check here, here and panicking in general, I finally figured out how to get my data back after some Hard core. Brute force. Raw file Reverse-Engineering. How? Step 1 : Find your raw data files. The tokufiles have an extension of ....

Kshitij Banerjee