I watched through some of this. This all seems very wise and well-informed. However the answer given is to spend a lot of energy attempting to work the system in fairly subtle ways to get the outcome you want - that the organization will accept your well-intentioned contributions to their functioning and eventual profit. All it takes is one or two people in the right place that don't share the same goal or are just really not that good to turn that into an exercise in futility.
I've worked at several startups now where the people in charge of A/B testing couldn't design a single A/B test that returned statistically significant results out of dozens of attempts each. Not. A. Single. One. Even the so called experts. And that's using their own calculations, which were already poorly designed attempts at p hacking.
The worst part is that some of the tests were still used to justify decisions to and by management. SMH
Is that the fault of the experiment? Or was it a weak manipulation and no-result was correct (e.g., changing the purchase button to a slightly different shade of green and expecting a higher conversion rate)?
If you are unable to provide your manager reasons for new development, he's going to find someone else to do your job, someone who will give him a report with charts and numbers that he can use to justify and expand his teams operational and headcount budget.
Giving a manager a report that says "there are little to no modifications that we can make at this time to improve UX" is a CLM for him.
IMO, this is still a failure to understand split testing. It is not to discover if changing a button color matters, but to explore the universe of all possible treatments and how they impact your most important business metrics.
It’s a global optimization problem, not a scientific way to understand how a specific change impacts users. Testers that have this mindset tend to be locally constrained and less likely to have bigger wins.
A no difference result in a well designed study is very valuable and can be used to justify decisions. It doesn't matter if we A or B so choose based on cost or the CEOs favorite color or a coin flip.
The phrase "social media" has become passé and even toxic.
The language around BTC & Crypto have political & other baggage.
Web3 is an attempt to re-package the past 10-15 years of web trends in a hip new label that leaves behind the baggage and allows for the appearance of "new stuff happening". But nothing much new is happening.
https://youtu.be/UFqz5xkgRbs