Phi-2 basically demonstrated that you don't need a very large model to figure out language. It not very smart but speaks perfect English. it's not obvious the best way to gain IQ is to have a larger language model. some other structure might be needed.
No, and this is covered in the various Phi papers, as well as TinyStories [0]:
> Models with around 125M parameters such as GPT-Neo (small) or GPT-2 (small) can rarely generate coherent and consistent English text beyond a few words even after extensive training.
I stopped using google map to navigate while driving due to the low contrast issue. I used to be able to scan the map and figure out which fork I should take. with the current google map, I have to stare at it for a fraction of a second to be sure. Now I use Apple map and works much better for me.
Stephen Wolfram (SW): Hello, Donald. Have you heard about OpenAI's language model, ChatGPT?
Donald Knuth (DK): I've come across some mentions, yes. AI has certainly come a long way since the Turing Test.
SW: Indeed. What's interesting is that it's less about designing an algorithm and more about generating one from massive amounts of data. It's quite aligned with my principle of computational irreducibility.
DK: In a sense, yes. Yet the allure of crafting an elegant algorithm, one that you can describe and understand fully, is something I still hold dear. The idea of a "black box" AI that we cannot comprehend is a bit disconcerting to me.
SW: That's a fair point. But sometimes the rules generated by the computational universe can create incredibly complex, yet coherent, behavior from simple inputs. Much like in cellular automata.
DK: It's interesting to think of it in that way. However, my perspective is more aligned with understanding and breaking down complexity. AI, in its current form, seems more about handling and even generating complexity.
SW: True, but even these complex systems are derived from rules and patterns, whether we can see them easily or not. It's a different kind of exploration into the computational universe.
DK: It certainly raises interesting questions for the future of computer science and AI. Whether it's rule-based systems or neural networks trained on vast data, we're still exploring the boundaries of what machines can do.
SW: Indeed. And ChatGPT, as it stands, is an intriguing example of that exploration. It's fascinating to see how this field evolves.
DK: Absolutely. There's always something new to learn.
Initially, I didn't see much value in the metaverse due to the high barrier to entry for end users and the limited network effect. However, I am bullish now. With the advancement of technologies like GPT-4 and Stable Diffusion, there is potential for more interesting NPCs in the metaverse. Additionally, Virtual Reality (VR) could serve as a superior user interface for these technologies compared to text-based interfaces like chatGPT. This could enable the metaverse to scale without solely relying on the network effect, as engaging NPCs and VR experiences can provide value to users even in the early stages of development, and continue to evolve and expand as technology progresses.
I can see how this end. Musk decided to do more experiments on monkeys. Soon monkey + chip become smarter than human. And human cease to exist shortly after.
I have been in finance industry for more than 20 years. People have been talking about replacing USD as reserve currency for as long as I can remember. The problem is nothing else is better. EUR? JPY? GBP? CNY? Gold? All of them have more issues compared to USD.
Model T is not efficient by modern standard. But it was the much more efficient compared to horse carriage at the time. Bitcoin is much less efficient than USD now. Very difficult to see why bitcoin would replace USD.
De-dollarization won’t happen in China. Chinese government is addicted to economic growth. The legitimacy of the government depends on it. There are only 3 ways to generate growth: infrastructure investment, consumption and export. The investment approach generates too much debt and one can argue China has reached limit there. Chinese has never been able to increase the consumption as a percentage of GDP in a meaningful way. So they have to rely on export. There is no way they can grow export without dollar.
Not quite - legally it's whatever is in the agreement and those have been (and still will be) subject to negotiation. In credit agreements without replacement language (e.g., pre 2018 agreements), a Borrower may very well be forced to borrower in prime, for instance.
What you're talking about is the credit spread adjustments from the ARRC formulation of the LIBOR replacement language (11.448 bps for 1 month, 26.161 bps for 3 months and 42.826 bps for 6 months) [0]. In reality, the credit spread adjustments for Term SOFR are still moving and the market has been all over the place - I've seen 10/15/25 bps at 1mo/3mo/6mo tenors (the most common formulation in the leveraged space right now) or a 10 bps flat adjustment (generally viewed as aggressive).
Completely tangentially related: I love basis points. Up until I started working in the FinTech I could not grasp the need for it... but after much repeating "x% plus 3%" and then having to clarify that is 300 basis points and not x*1.03, it got my eyes opened.
It's also useful for negotiating rates. Saying you want a rate to be .2% lower is awkward and wordy. Saying 'can we take it down 20 bips?' sounds much more pro ;).
26bps was the recommended adjustment based on historical analysis at the time. The actual spread in the market is not a constant, and has been below 26bps for a long time. The current LIBOR-SOFR spread is around 12bps. It's all negotiated, so people who earn interest are arguing it should be 26bps and people who pay interest are arguing it should be 12 bps. The market seem to be settling on 3M LIBOR = 3M SOFR + 15-20bps.
