Bookmarks
Dissipative Adaptation: The Origins of Life and Deep Learning
The document explores the concept of Dissipative Adaptation, drawing parallels between the emergence of life and the mechanisms of Deep Learning. It discusses the work of Jeremy England and his theory of non-equilibrium statistical mechanics known as Dissipative Adaptation, which explains the self-organizing behavior of Deep Learning. The text delves into how neural networks evolve through training, emphasizing the role of external observations in driving the system towards minimizing entropy. It contrasts the mechanisms of Dissipative Adaptation with current Deep Learning architectures, highlighting similarities in alignment of components to maximize energy dissipation or information gradient.
Re: [Fis] A PROPOSAL ABOUT THE DEFINITION OF INFORMATION
The email exchange discusses the concept of negative entropy and its implications in mathematics and thermodynamics. Sungchul Ji questions the validity of negative entropy based on the Third Law of Thermodynamics. Arturo Tozzi argues for the existence of negative entropy in certain cases and relates it to information theory and free energy.
Information
The text discusses the challenges and complexities of measuring and quantifying information, particularly in terms of storage capacity, compression, and entropy. It explores various examples, such as genome information, human sensory capabilities, and the information content of objects like water molecules and black holes. The relationship between information, entropy, and physical properties is also highlighted.
Bekenstein bound
The Bekenstein bound is an upper limit on the entropy or information that can be contained within a given finite region of space with a finite amount of energy. It implies that the information of a physical system must be finite if the region of space and energy are finite. The bound was derived from arguments involving black holes and has implications for thermodynamics and general relativity. It can be proven in the framework of quantum field theory and has applications in various fields, such as black hole thermodynamics and the study of human brains.
Temperature as Joules per Bit
The paper suggests that temperature should be defined in terms of entropy, rather than vice versa. It argues that the current practice of measuring entropy in joules per kelvin is a historical artifact and proposes measuring entropy in bits instead. The paper also discusses the role of information in thermodynamics and the thermodynamic cost of erasure. It concludes by suggesting that entropy, not temperature, should have its own unit and that Boltzmann's constant should be dissolved.
Subcategories
- applications (15)
- computer_architecture (1)
- ethics (1)
- expert_systems (2)
- game_ai (5)
- knowledge_representation (4)
- machine_learning (324)
- natural_language_processing (3)
- planning_and_scheduling (2)
- robotics (2)
- software_development (1)
- theory (1)