Book: 20220711 to 20221130, "Noise" by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein

20220718 - Introduction: Two Kinds of Error 3

Bias is something we can try to overcome(only reduce, not completely remove). Noises are more like randomness.

This book reminds me of How to Measure Anything by Douglas W. Hubbard. If we can measure it, then surely we can reduce bias and noises. Apart from that, there must be something methods that can help us to reduce noises without measurements.

Part I Finding Noise 11
20220722 - 1 Crime and Noisy Punishment 13

Noise is everywhere. Everyone do it everyday. What is right? No such thing.

This reminds me of quantum effect. Material is not rigid, it is possibilities. When someone observe it, it changes.

How to handle it? Focus on the macro level. Set up rules. Specific details are not that important.

20220727 - 2 A Noisy System 23

Noise is unwanted variation. Variation is the sympton of diversity, which is critical for natural selection.

But variation also means huge waste. If we can remove obvious wrong answers, that would hugely reduce the cost (of time, resource, energy, etc.) of revolution.

One interpretation is enough, and we experience it as true. p31

Unless our interpretation is only about objective factors, such as science and engineering.

Our confidence is not always beneficial. If the judgement is involved with many external subjective factors, then our experience helps little. This is also explained in "Thinking, fast and slow". Surgeon and chess professional has good reason to be confident, but finance analyser and judges don't.

20220802 - 3 Singular Decisions 34

For singular decision, noise audit doesn't work. Instead, we need to analynize its cause.

We need to analynize it based on objective measurement, instead of our subjective experience. So the first principle works here. Inaccurate measurement is much better than no measurement.

Does quantum mechanism work here? Yes. Chain effects are always there, which make it impossible to find the very best choice. We don't need best choice, and only need good choice.

Make choice right is more important thant make right choice. Compound effect decides the result after we make the choice.

There is no way to make very accurate measurement. What we need is roughly accurate measurement.

If accurate measurement is not that important, then what is critical? Just like Quantum mechanics, we should not focus on a single particle, instead, we should observe from higher level, to measure the collective features of the matter.

Measurement of aggregation is also not very accurate, and it doesn't need to be so accurate either.

Part II Your Mind Is a Measuring Instrument 39
20220806 - 4 Matters of Judgment 43

Noise is critical to measurement. Predictive or evaluative.

Just like the stock rankings from Wall Street analysts, the middle value reduce noise, but the bias alone makes it useless. But at least we get an anchor.

SP500 index is another anchor. As it contains everyone in the market, it seems much more reliable. Then it's bias. SP500 index cannot tell us whether a bubble is coming.

Can the history of SP500 index tells us more about reality? Is inertia there in the trend of SP500 index? Economic growth surely has inertia, how can we predict its trend of the next 10 years? Population, innovation, social stability. These factors affects economic growth.

What should we do if we have to make a bet based on very limited information? Ignore those information and go to the mean value. To stock market investment, this means invest in SP500 index fund. To some extent, those limited information are not really information, but distractions.

Making money through the Black Swans in stock market, is actually making money from other investors' overconfidence. It needs careful calculation and patience. Because most of people are almost always overconfidence, it's possible to make money that way.

20220810 - 5 Measuring Error 55

Bias and noise are both error. They casue same level negative consequence. However, because most of people understand the damage caused by bias(so did a lot to minimize it), noise may cause more loss.

People are very keen to get perfect hits and highly sensitive to small errors, but they hardly care at all about hte difference between two large errors. p65

It's easy to notice that something is imperfect, but hardly we notice that something is terriablly wrong. Imperfect is normally in short term, and big problem is normally in long term. Because of our gene drawbacks, we don't pay much attention to long term crisis.

If there is less noise, then it's much eaiser to reduce bias.

The error equation does not apply to evaluative judgments. p67

This is the reason that Wall Street analysts cannot predict the share price of any specific stock.

20220814 - 6 The Analysis of Noise 69

Level noise and pattern noise are both system noise. Level noise and pattern noise contribute same effect to systen noise.

Level noise is more like individual bias. Pattern noise is unique to each person.

Occasion noise is part of pattern noise. The judge may not agree with himself/herself.

20220817 - 7 Occasion Noise 79

Occasion noise only cause 18% differences. Not much, but I believe it heavily depends on the time gap and the intentionly training between tests.

Our brain changes as time goes by. It changes more with training, study, communication, etc.

So, before making any critical decision, we should discuss it with other people, and don't act too soon.

20220824 - 8 How Groups Amplify Noise 94

Choice, decision, judge, etc. are all events. They are affected by bias and noises.

Compound effect is about effort, which lasts for long time. It's not affected much by bias or noises.

So, make the choice right, always beat "make the right choice". Habit and effort most likely beat luck. Action is more important than position. How about meditation?

Social pressure is decided by our gene. There is a good reason for that. The first one who express his/her opinion normally are quite confident. This confidence normally comes from good foundation. For other people, if they are not sure which decision is better, it's safter to follow the first one.

However, the first one could be wrong, then it leads to big mistakes, or at least leads to the loss of group wisdom. This is why it's critical to allow everyone to express their opinion independently. Should we force everyone to vote? Surely yes.

Group amplify noise is like resonance. How to reduce it? Let facts and numbers speak, and let everyone speak without concern.

Where does resonance come from? Entropy increases all the time, but not always along the fastest way. Resonance slow it down.

Part III Noise in Predictive Judgments 107
20220829 - 9 Judgments and Models 111

You know most of the what you need to know to assess the two cases, but gazing into the future is deeply uncertain. p115

Simple models beats humans. p116

Any data is better than no data. Any rule is better than no rule.

Human make decision based on their intuition, which means no data and no rule, only feelings.

Is it possible the problem is caused by lacking of "skin in the game"? Can parents predict their children's future before they get independence?

Then it's about stock market investment. Most of people make decisions based on their intuition. That's the source of disaster. Our intuition is only reliable with objective situations.

We also should not trust complicated economy math model, because it try to solve a problem with so many people get involved.

Replacing you with a model of you does two things: it eliminates your subtlety, and it eliminates your pattern noise. p120

Write down the reasoning may reduce subtlety. Review the analysis later on may reduce pattern noise.

20220905 - 10 Noiseless Rules 123

Improper Linear models (giving all the predictors equal weights): All you need is a collection of predictors that you can trust to be correlated with the outcome. p124, p126

Most of the sampling data is too small. So, any weight in only proper in specific dataset. Improper Linear models are more accurate overall.

Broken-leg principle is the reason that AI can beat frugal model. It recognizes exceptions.

People want to be in charge. They enjoy the feeling when making decisions. So, models and AIs should always be used as optional tools.

Then how about Robotaxi? People can choose to take a Robotaxi, or a human driving taxi/uber. Just like computers. We can choose paper and pen if we want.

20220911 - 11 Objective Ignorance 137

We don't know what's going to happen, and we don't have complete information about what had happened. So we cannot predict the future.

But there is something we can predict. For example, CPU will get much faster in the next 10 years, and sustainable energy will replace fossil energy in the next 30 years. What's the difference about these predictions? They can be evaluated through the first principle or data.

Why people perfer to follow their intuition instead of model analysis? They like the feeling of in charge. So it will take many years for self-driving to replace human drivers.

Following models means we admit that what we see are distorted view. Our eyes and mind are not reliable

Models are only slightly better than our intuition. Compound effects brings us huge difference. And, when we rely on models, we will try to gather more information and do more objective analysis to improve the accuracy. From the other side, we cannot improve our intuition.

20220918 - 12 The Valley of the Normal 148

The relationship between shoe size and mathematical ability is a perfect example to explain the difference between causation and correlation.

The first principle can only solve problems when no intention get involved, such as math, chess, physics, but not financial market, voting, education, etc., unless it's about long term trend.

Is "Gun, Germs and Steel" correlation or causation of human civilization evolvement? How about energy(affecting Gun, Steel)? Short term fluctuation is not predictable, but long term trend can be predicted by the first principle.

We always prefer to find causal explanation for an event, but what we find is most likely an illusion. Statistical explanation needs our deliberate effort (system 2) to think about it. That's tiring.

Part IV How Noise Happens 159
20220924 - 13 Heuristics, Biases, and Noise 161

A heuristic for answering a difficult question is to find the answer to an easier one. p166

Between right (but hard) solution and wrong(but easy) solution, we normally choose the latter. That sacrificed long term benefit for short ter benefit.

Base-rate neglect: taking the outside view can make a large difference and prevent significant errors. p167

This is part of the reasons that we should choose SP500 index fund instead of choosing individual stocks.

Anchoring effect is used in "Secrets of Power Negotiating by Roger Dawson". This is part of Conclusion effect: we draw our conclustion first, then gather evidences to support it.

Halo effect is a major problem for me. I always jump to conclusion quickly, then it's not easy to change it.

Again, the First Principle is the best way to resist these three biases.

20220930 - 14 The Matching Operation 176

"Life is never as good or as bad as one thinks." ― Guy de Maupassant

Regression toward the mean, unless there is a Black Swan.

For safe bet, extreme outcomes are followed by more moderate ones.

What's the difference between noise and bias? Individual, temporary bias is noise.

Taking the outside view means anchoring your prediction on the average outcome. p183

When serious judgment is necessary, the outside view must be part of the solution. p183

How to judge things if there are more than 7 levels? We can put them into major categories first, then split the things in the same major category into small categories.

The better way is to figure out the features that decide the level. Give each feature a score, then sum up all scores as the measurement.

20221001 - 15 Scales 187

People are good at judging based on comparison. They know which one is better, worse, more intense, etc. But they have no idea about the absolute value associate with a specific case.

So we need to figure out the anchor before making decision. For example, the price of a property, or salary, or the price of some jewellery.

For jurors, they know the defendant is crimial or not, but they have no idea how long the defendant should stay in jail if he/she is convicted.

20221010 - 16 Patterns 200

Bias: wrong decision which is caused by inaccurate perception.

Level noise: different people have different opinion about the same event.

Stable pattern: same person has different opinion about the same event under different context.

Occasional pattern: same person has different opinion about the same event under same context.

20221011 - 17 The Sources of Noise 210

Why our gene give us so much noises? Noise is the also the source of creativity. We need huge amount of varieties to evolve, although most of those varieties are errors. No way to figure out which "error" is correct, before the verification.

Causes are natural; statistics are difficult. p219

Stay in the present moment is a false description of the problem. What we really need to do is paying attention all the time. That must help a lot with noise reduction.

Part V Improving Judgments 221
20221026 - 18 Better Judges for Better Judgments 225

Superforecasters? p225

This is disappointing. There is no "superforecasters". They are more likely to make correct prediction because of luck.

Respect-experts? p226

Disappointed again. If the expertise is about objective stuff, such as science, chess or art, those respect-experts are real experts. Or else, they are not, even if they got great achivement in the past. For example, economists, politicians, stock traders.

Do the one with higher intelligence and open-minded make better decision? If we don't have other alternative way to choose, then yes. But if possible, we should stick to the first principle: try to analyze things objectively.

20221027 - 19 Debiasing and Decision Hygiene 236

Can decision observer help to improve the prediction of stock market trend? Yes. A simple checklist is much better than our instinction to reduce biases. For something not so complicated (less factors get involved), it works much better. Checklist is a type of first principle tool.

Decision hygiene should be a habit. Like washing hands, it helps us through compound effects. We don't know when it takes effect, but it surely help us living longer and happier.

20221105 - 20 Sequencing Information in Forensic Science 245

What everyone see is distorted reality. It's the imagination in their mind. Include the best experts and professionals.

Any unnecessary information is distraction, which may cause noise and bias.

20221106 - 21 Selection and Aggregation in Forecasting 259

2% are superforcasters? I don't buy it. The world is too complicated. There is no way to forcast the future, or else they can make really good money in stock market.

Warren Buffett made good money by excellent analysis, not forcasting. The best way to forcast is to make it happen.

The way to find the superforcasters is flawed. If let them try again, I think most of them would fail. The world is full of black swans.

However, keep openmind and analyze objectively, helps to forcast a bit. Not much.

How to choose team members to make a good team for judgement?

1. Wise 2. Complementary to each other 3. Indepedent to each other

20221112 - 22 Guidelines in Medicine 273

In general, guidelines split the process into many steps which can be measured objectively or less subjectively. The result of aggregation is less noisy.

However, psychiatry is exception, because we can only measure through subjective responses from patients. If there is nothing special in the brain scan, we have to make decision based on subjective measurement. Some people can be talked through, but some needs medicine. In many cases, we have no idea which one is better.

20221112 - 23 Denning the Scale in Performance Ratings 287

How does Elon Musk overcome the bias and noises in performance evaluation?

I guess:

1. Hire the one with necessary knowledge and moral, and willing to think instead of following habit and subconsciousness.

2. Make sure everyone know the aim.

3. Evaluate performance based on individual achivements.

20221119 - 24 Structure in Hiring 300

After all those improvements (decomposition, independent assessment, delayed holistic evaluation), the success rate rise from 58% to 67%. Not bad, but far from satisfying. Google is still falling.

Maybe, once above certain threshold, interviewing is not that important. The critical part is, what they are going to do after joining the company. That part makes SpaceX and Tesla so special.

20221119 - 25 The Mediating Assessments Protocol 312

This reminds me how Cathie Wood assess the value of a company. It's better to let different team analyze different aspect of a company, so the teams can be independent to each other. (decomposition and independent assessment)

We should use the mediating assessments protocol whenever possible. This can only improve the judgement for around 9%, but under compound effect over many years, that would cause huge differences.

Study and practice can cause much more differences.

Part VI Optimal Noise 325
20221125 - 26 The Costs of Noise Reduction 329

Algorithm can be far less imperfect han noisy and often-biased human judgment. p337

Because we can check the algorithm easily, we have better chance to keep improving it. The cost is normally not real problem.

20221125 - 27 Dignity 339

Everyone, every case is special. To judge someone or something fairly, we need to do a lot of investigation. That cost is too high.

The purpose of guideline is to reduce noise and bias. Or, improve productivity. Many people don't mind following rules to handle work, but many people need the "I am in charge" feeling, or else they look for all opportunities to break the rules, such as Mr Richard Feynman.

Philip Howard is right: we need to set up general principles. This is what Elon Musk does: no rigid rules, and welcome everyone to challenge existing rules. If got support from other people based on common sense, then give it a try. If failed, then roll back to the previous rule, or continue to improve the new rule.

To some extent, we can adapt this strategy to most of tasks.

20221126 - 28 Rules or Standards? 350

Rules have an important feature: they reduce the role of judgment. p351

When standards are in place, judges have to do a lot of work to specify the meaning of open-ended terms. p351

With correct goal setting up, we can reduce the noises and bias with affordable cost. For legal system, the goal is not  justice or fairness, but a better guideline. We need to keep improve the rules and standards in the guideline, or else follow it. And, improvement doesn't mean more complicated or cover more aspects, it means less cost and better effect.

20221130 - Review and Conclusion: Taking Noise Seriously 361

Noise and bias exist in judgement. Judgement is subjective. It is based on narratives.

So, noise and bias come from narratives.

Noise is as bad as bias. Why is that? It's like the precision of missle. There is bias and there is noise. The result is same: they let the missle miss the target.

The goal of judgment is accuracy, not individual expression. p371

We should make it as objective as possible.

The major problem of noise is that people don't realize how serious the problem is, and almost no one  would appreciate the effort of reducing noise.

20221130 - Epilogue: A Less Noisy World 377

Let's wait for AGI.    :-)

Appendix A How to Conduct a Noise Audit 379
Appendix B A Checklist for a Decision Observer 386
Appendix C Correcting Predictions 389