AI is dataset

Recent Insights on AI Theory

The framework of an AI model is not as important as previously thought. Different networks can converge to similar results when trained on the same dataset.

This challenges the traditional view that a specific architecture is required for achieving a certain level of intelligence. For example, someone I know mentioned that CNNs can also be used for AGI (Artificial General Intelligence), which is a plausible idea.

The Phi-3 experiment by Microsoft has further demonstrated that models like GPT-4 and GPT-3, which are trained on vast amounts of human-generated text, may be overfitting to noisy data.

This raises an important point: when we refer to an AI model, such as GPT-3, GPT-4, or Gemini, we are essentially referring to the dataset that it was trained on, rather than the model itself.