What is inductive reasoning?

Why do we put so much faith in a fallible process?

Premises
All 𝛂 are 𝛃.
𝑥 is 𝛂

Conclusion
𝑥 is 𝛃.

This conclusion is true.  

Unless the premises have been derived from data, in which case, I have no idea.

In recent years, the idea of being data-driven has become synonymous with objectivity. This remains a point of frustration for me, as an inference from data requires inductive reasoning, a fallible process that does not guarantee a true conclusion. Inductive reasoning is less objective than its most obvious counterpart, deductive logic, which has much closer ties to traditional domain expertise.

Deductive logic is a process that starts from a set of rules (premises, domain expertise, generalizations) to inform a specific example (data, future prediction). Inductive reasoning is the reverse, going from specific examples (data, past observations) to the general rules (beliefs, hypotheses, rules generalizations, premises).

In deductive logic, one assumes a premise to be true.  A conclusion can be valid (true) if it is derived from the premises, and the argument is sound.

In inductive reasoning, there are no assumptions. Instead, there are beliefs — a set of premises with variable confidences. Some beliefs are more powerful (more likely to be true) than others.

Practically, what does this look like? If a deductive premise is expressed as “all 𝛂 are 𝛃,” analogous inductive premises could take the form: “there is good reason to believe all 𝛂 are 𝛃,” or “every known example of 𝛂 has also been 𝛃,” or “many many 𝛂 are also 𝛃.” A conclusion from inductive reasoning is one that is likely or probable, given the uncertainty of the beliefs. Nothing is ever proven, certainly not objective. It always depends.

In the specific example above, one cannot necessarily conclude that 𝑥 is 𝛃 because beliefs (unlike assumptions) can be inconsistent. There may already be high confidence in the belief that that 𝑥 is not 𝛃. The process of choosing between possible (at times contradictory) beliefs and updating confidence in them as new data is observed inductive reasoning. Particularly in its most advanced forms, it is more art than science.

In the 𝛂, 𝛃, 𝑥 example above, what can one conclude if the premises are derived from data?  It depends on why the question was asked, how the beliefs were generated, and even what actions might be taken as a result. So, as I began, I have no idea.

Abstract ideas like this benefit from concrete examples. This will be the focus of the next post (and probably dozens after it).

First, let’s make sure we set the stakes sufficiently high.

This is not just a semantic game for philosophers. Decision-makers who act confidently based on data alone will overestimate the strength and ’ strength and quality, risking the success of long-term goals. This impacts every field from doctors practicing medicine to entrepreneurs and executives setting business strategy, investors hedging the market, and government officials deriving policy.

I have found that understanding the nature of inductive reasoning, its benefits, and its disadvantages allow me to use it as a tool more effectively. That is what this series is about. There are limits of inference. Data is not magic.