Entropy in machine learning and information theory refers to the uncertainty or randomness in a random variable or dataset. Here are the key things to know about entropy:

• It is a measure of impurity in a collection of examples. Pure nodes have zero entropy while mixed nodes have high entropy.

• In binary classification, entropy is maximum (1) when the class distribution is 50-50 and minimum (0) when all examples belong to a single class.

• The formula for calculating entropy (E) of a collection S with probability of elements p(x) is:

E(S) = -Σ p(x) log2 p(x)

• It is used as a metric for decision tree algorithms like ID3, C4.5, CART etc. to select the best attribute to split the node on.

• The attribute with the greatest information gain (highest reduction in entropy) after splitting is chosen as the splitting criterion.

• Low entropy nodes/partitions mean examples are well classified with high certainty.

• Thus reducing entropy helps generate Pure leaf nodes, improving predictive ability.

So in summary, entropy quantifies the uncertainty in data, and reducing it helps machine learning algorithms make smarter predictions with higher confidence.