Entropy
Entropy is all about randomness and unpredictability.
Meaning of the word Entropy
In Greek, En means in and trop means turning. Together they mean, turning towards its final state. This word was meant to describe the fact that things always drift towards disorder.
For example, a bookshelf once sorted, will eventually end up being mixed up. The heat on the stove will eventually spread to the entire room.
Other similar words are tropical, phototropism and many more.
Macro view vs Microstate in Entropy
- Macro view. This is the big picture state you see. For example, a bookshelf is either messy or neat.
- Microstate. These are the small details inside the macro view. For a messy shelf, many layouts are possible. For a random file, many character patterns are possible.
High Entropy vs Low Entropy
In a high entropy system, the microstate has a vast number of combinations possible that represents the same macro view.
Entry as Noun vs Verb
The word Entropy is used in two different ways.
- noun - State where the highest number of possible microstates exist.
- verb - Transformation from low to high level of randomness.
Entropy in Information Technology
We come across the word Entropy in multiple places. Some of the most important ones are listed below.
- Security - High Entropy means high randomness. Used in secrets.
- Software Entropy - Software rots over time to a bad state.
- Compression - If the original file has low entropy, then compression yields better results. Also, after compression, the generated file has high entropy.