Hand-picked selection of articles on AI fundamentals/concepts
Overview
- Here’s a hand-picked selection of articles on AI fundamentals/concepts that cover the entire process of building neural nets to training them to evaluating results.
Algorithms/Architecture
- Linear and Logistic Regression
- k-Nearest Neighbors
- Clustering
- Support Vector Machines (SVM)
- Naive Bayes
- Decision Trees and Ensemble Methods
- ML Algorithms Comparative Analysis
- DL Architectures Comparative Analysis
- Prompt Engineering
- Generative Adversarial Networks (GANs)
- Diffusion Models
- Graph Neural Networks
- Attention
- Separable Convolutions
- Inductive Bias
- Convolutional Neural Networks
- Reinforcement Learning
- Mixture-of-Experts
- State Space Models
- Agents
- FlashAttention
- Model Acceleration
- Speculative Decoding
- Cross Validation
Data/Training
- Data Sampling
- Data Imbalance
- Standardization vs. Normalization
- Learning Paradigms
- Xavier Initialization
- Padding and Packing
- Regularization
- Gradient Descent and Backprop
- Activation Functions
- Loss Functions
- Activation Functions
- Fine-tuning Models
- Splitting Datasets
- Batchnorm
- Dropout
- Double Descent
- Fine-Tuning and Evaluating BERT
- Training Loss > Validation Loss?
- SVM Kernel/Polynomial Trick
- Bias Variance Tradeoff
- Gradient Accumulation and Checkpointing
- Parameter Efficient Fine-Tuning
- Hypernetworks
- Distributed Training Parallelism
Speech
Vision
- Vision Transformer (ViT)
- Receptive Field
- Residual Networks/Skip Connections
- GPT-4o Native Image Generation
NLP
- Word Vectors/Embeddings
- NLP Tasks
- Preprocessing
- Tokenization
- Data Sampling
- Neural Architectures
- Attention
- Transformers
- Token Sampling Methods
- Encoder vs. Decoder vs. Encoder-Decoder Models
- Overview of Large Language Models (LLMs)
- Reinforcement Fine-Tuning
- Preference Optimization
- Machine Translation
- Knowledge Graphs
- Hallucination Detection and Mitigation
- AI Text Detection Techniques
- Named Entity Recognition
- Textual Entailment
- Retrieval Augmented Generation (RAG)
- LLM Context Length Extension
- Document Intelligence
- Code Mixing and Switching
- Large Language Model Ops (LLMOps)
- LLM/VLM Benchmarks
Multimodal
Models
- BERT
- GPT
- CLIP
- Meena
- ChatGPT
- GPT-4
- LLaMA
- Alpaca
- Gemini
- Toolformer
- Visual ChatGPT
- TaskMatrix.AI
- BigBird
- OpenAI o1
- DeepSeek R1
- DeepSeek Janus-Pro
- Gemma 3n
Offline/Online Evaluation
MLOps
On-Device AI
- Model Compression
- Personally Identifiable Information (PII)
- Federated Learning
- Differential Privacy
- On-device Transformers
Project Planning, Scheduling, Execution
Miscellaneous
- Ilya Sutskever’s Top 30
- Debugging Model Training
- ML Runtimes
- Chain Rule
- Bayes’ Theorem
- Probability Calibration
- Multiclass vs. Multilabel Classification
- N-Dimensional Tensor Product
- PyTorch vs. TensorFlow
- Approximate Nearest Neighbors – Similarity Search
- Transferability Estimation
- TensorBoard
- Convolutional Neural Networks for Text Classification
- Relationship between Hidden Markov Models and Naive Bayes
- Maximum Entropy Markov Models
- Conditional Random Fields
