LLMs are strangely-shaped tools

few hit AI apps exist because LLMs are what i’d call very “strangely-shaped” tools.

most tools are built with a specific purpose in mind like a screwdriver or a car.

but LLMs were something we stumbled upon by predicting text and playing with RL – we didn’t design the shape of them beforehand, we just let them naturally evolve into what the loss optimized for.

it’s clear they’re good at many things, but they are so strangely shaped that it’s easy to fall into traps when making products.

AI agents are a good example of a trap (for now), where it’s easy to spend months trying to perfect your scaffolding yet never quite reaching the level of reliability you’d hope for.

long-term memory implemented solely via RAG is another trap. it’s just tempting enough to try, but the results aren’t as good as they should be.

other common inadequacies include poor search, hallucinations, and high inference costs. but there’s a long list of subtle weaknesses which few tinkerers ever notice as well as many weaknesses (and strengths) which remain unfound.

much of the frontier of LLM posttraining is currently concerned with these inadequacies – wondering how we can mold these strangely-shaped LLMs we have grown into a slightly more suitable form for the problems we face.

this is hard, even for the major labs. as we slowly progress on it, i’d expect to continue to see most AI products attempt to solve the same problems via the same methods, further suffering from lack of distinctness both in performance and aesthetic, because they don’t have the right connection between research teams and product teams (or perhaps the right vision to begin with).

it’s telling that among the few recent consumer successes like midjourney or perplexity, competitors are hyper-focused on directly copying winners rather than exploring the vast new frontier of things which could be built instead. this makes sense because the frontier is strangely-shaped, as a result of the underlying catalyst itself being strangely-shaped.

it’s not uncommon for services to launch a feature literally called “AI” which is primarily composed of literal magic wands and glitter emoji simply because the product designer has no idea how to actually convey the intended experience to the user. 2024 is certainly not a year one would be fired for using too much AI.

I expect it to get more interesting later this year and especially in 2025, but it’s still been a surreal experience continually contrasting my day to day life in san francisco with that of the actual real world (note: SF is not real in this example).

the above is also relevant to some of the reasons i have longer agi timelines than i did a few years ago. agi is not a strangely-shaped tool. in fact, it is quite literally the opposite.

Originally posted via Twitter/X, but mirrored here for convenience.

If you liked this post you may be interested in the rest of my website too!

Music

This post has a bunch of unordered youtube videos with music I like.
Some of the videos are niche so hopefully you find something new you like!

Logical Emotion: トライアングルどら息息子


Dimash Qudaibergen: SOS d’un terrien en détresse (Piet Arion Cover)


Polyphia: Playing God


Kevin Penkin: Hanezeve Caradhina [Made in Abyss OST] (Takeshi Saitou Cover)


Deemo: Marigold (ふぃくしのん / phyxinon cover)


Nobuo Uematsu [FFXIV OST]: Great Gubal Library (Hard) Theme


Masayoshi Soken [FFXIV Endwalker OST]: Piano Covers by SLSMusic


Tosin Abasi: Thump


Keiichi Okabe [Nier Automata OST]: 壊レタ世界ノ歌 / Weight of the World


Shibayan Records: Fall in The Dark


Xi: Anima


Clowncore: Computers


Animals as Leaders: Behaving Badly


Coprofago: Motion


Gojira: The Art of Dying


狼と香辛料 (Spice and Wolf: 旅の途中 (OP)


Bobby Jarzombek: Selected Drum Solos


Infected Mushroom: Dancing With Kadafi


Ashleigh Bridges (OSRS OST): Coil


ダンベル何キロ持てる?: お願いマッスル (OP)


Meshuggah: New Millennium Cyanide Christ


Keiichi Okabo (NieR Automata OST): 遊園施設 (Amusement Park)


Iconoclasm: perditus†paradisus


Nobuo Uematsu [FFXIV OST]: The Lost City of Amdapor (Hard) Theme


Uneven Structure: Awe


Shiho Fujii, Atsuko Asahi, Ryo Nagamatsu, Yasuaki Iwata [Mario Kart 8 OST]: Dolphin Shoals


UNDEAD CORPORATION: The Revolution


Indricothere: III


Nekomimi Syndrome: Fur War, Pur War


All That Remains: Six


An: Sadistic Confusion


Andy James: Burn it Down


Kevin Penkin (Made in Abyss S2 OST): VOH ft. Takeshi Saito


Free Alpha By Design

Complex systems of life often contain a multitude of shortcuts: sources of alpha which, should you choose to exploit them, give notable advantage. Many classes of these shortcuts are features rather than bugs, and it is no accident of many systems that only a select minority are able to tactfully navigate them.

In many cases there exists zero-sum shortcuts, which, if all of society were to adopt them tomorrow, would have their effectiveness instantly curtailed to zero. Luckily there also exists many which are positive-sum and benefit both parties involved. This post contains a few notes and examples from both classes.

Venture Capital

In venture capital, it is strongly preferred that one receives an introduction to an investor prior to pitching them. This is, in general, a much easier way to meet with many classes of professionals than a cold-email.

At first glance this may appear shallow or nepotistic, but it’s more accurate to view it as one of the first tests to becoming a successful founder. If you’re unable to find any way to get an introduction to a person, it is more likely that you may also have difficulty with other similarly important roles that a founder must perform such as recruiting, sales, management, and further fundraising.

This warm-introduction requirement therefore functions less as “this person happens to know someone”, and more as “this person has the right set of traits in order to acquire this warm introduction, which we consider a modestly bullish investment signal”.

Due to the generality of this phenomenon, this paradigm repeats itself in many categories of society.

Job Hunting

It is much easier to get a job somewhere by messaging someone at that company and asking them to help you out, preferably with some baseline level of evidence that you are at least reasonably competent and aligned with the organization.

This may seem obvious, but at any given moment there are millions of reasonably-intelligent individuals bulk-sending their CVs to hundreds of unsuspecting organizations, when their time may be better spent narrowing their hit list down to only a select few companies, then delegating several hours of their time towards each of them.

You can go much further than this, of course. To give an example: the few times I’ve had an employer other than myself were because I specifically asked to be hired for a role I made up. Rather than seeing an available position on a job posting, I looked at the intersection between the needs of the organization and my own abilities, and proposed that intersection directly to the CEO as my job description. Many find this to be much more agentic than something like “asking the neighbor with overgrown grass if he will pay me $20 to mow it for him”, but the exact same thing is happening in both of these scenarios. Negotiating your salary is in a similar category of actions.

The reason why the above two sections describe intentional shortcuts is because they’re positive-sum. Negative-sum shortcuts are often unintentional (for example, stealing merchandise from a store), and zero-sum shortcuts can be of arbitrary intentionality.

User Support

Many are familiar with the basic techniques of escalating one’s user support channel with a large corporation to an agent which is both more responsive and more capable. Some classes of individuals may formalize this as e.g. “I’d like to speak with your manager”, but there’s often more effective methods depending on your use case. Patrick McKenzie has many good examples among his blog posts, and I’ve included a wonderful excerpt from him on banking and credit reports below.

While the above example is aimed at individuals that have been wronged by large financial institutions, many of the principles apply more generally.

On occasion, when I’ve had a particularly egregious and frustrating issue with a large company, I’ve emailed an executive in the relevant department with concise and kind language explaining the issue, where its alleviation is in the self-interest of my counterparty. This can sometimes be done all the way up to the CEO, but results vary depending on who both parties are, what the issue is, and how it is communicated (this is, however, a skill you can very much learn and perfect).

I’m fortunate to have a twitter account with a large amount of founders and investors in my following, which affords me the luxury of sometimes having CEOs fix issues I complain about. You don’t need a twitter account for this though; there are many executives and engineers that read forums like e.g. Hacker News and will directly respond to the right classes of issues on occasion.

Every Day Life: Flying

If you spend time thinking about the systems you regularly interact with, you can forecast some potential shortcuts by modelling both the incentive structures of the arbitrators of the system as well as the behavior of the average system participant.

Simple example: I fly pretty frequently. I often do things like:

  • Changing my seat an hour before takeoff to get two or three seats to myself. Laying down and taking up a full row is a much better experience than first class! This is done by combining 1) last-minute seat selection with 2) picking flight times where the flight is likely to be <80% full (which is unfortunately not a given for an arbitrary itinerary)
  • If I’m feeling particularly voracious when flight attendants decide to hand out biscuits, I might ask for three of them. I’ve never had a response of ‘no’ as there’s no reason for them to decline. I never ask for additional pretzels, should my choice of airline condemn me to such a fate.
  • If I want to board earlier than my boarding class, this is generally not scrutinized. I don’t abuse this, but sometimes it is simply more convenient both for myself and others (one reason it can be positive-sum is that I’m particularly quick both with my movement and with handling any luggage). Some airlines are okay with you taking up a first-class seat without paying for it, should one be empty.
  • Should the airline lose you your flight (or they require a set amount of passengers to switch flights), the compensation offered to is often highly negotiable (as most things in life are!), and can take many forms (flight credits, class upgrades, lodging, etc).
  • If you fly frequently in the US you should get TSA pre-checked. The advantage is more “reduces the variance and maximum time through security” rather than “reduces the average time through security”, as the former allows you a significantly more generous grace period. It’s also worth looking into credit card optimization if you travel frequently, but I’ll consider this outside of the scope of this post lest it turns into the equivalent of “they don’t want you to know this but you can bring candy inside of the movie theater”.

Exercise for the reader

Think about a system you regularly interact with and take some time writing down the incentives of the actors within the system. Think about actions which could be taken which are not frequently given as examples to follow, but which nonetheless match the ‘desires’ of the system.

It will be easier to find zero-sum examples in systems which are less heavily-optimized by market structures, e.g. searching for a secret to getting rich by trading derivatives better than anyone else will probably result in disappointment, whereas acquiring multiple snacks while on a flight won’t be a problem. I’d suggest picking something fun which is a quotidian yet mostly unscrutinized part of your daily life.

Try to come up with novel ideas, ideally by writing down things that come to your mind with as few stimuli to distract you as possible. I find this to be a really good exercise both in critical thinking and in agency.

If you enjoyed this post you may like other posts or my Twitter account. Thanks!

Optimal Webcam Setup

If you have important video calls (for example, you fundraise as the CEO of a company), you should have a good webcam setup. This post contains everything you need for one, at a cost of $700-$2,000, depending on which options you choose.

I. Camera

Most webcams are not very good, and you’ll want a real DLSR camera instead, optionally with a better lens:

Camera: Sony Alpha ZV-E10 ($800 with lens, $700 without)

Lens: Sigma 16mm f/1.4 Lens ($380)

II. Microphone

Although cheaper microphones work, it may be nice to have a high-end microphone which doesn’t get in your way:

Microphone: Rode NTG4+ Shotgun Microphone ($340)

III. Camera Accessories

Buy a power adapter for the camera so you never have to charge it: F1TP AC-PW20 AC Power Adapter ($25)

Buy a capture card for the camera for the best video quality and lowest latency: Elgato Cam Link 4K ($90)

IV. Mounts

You probably want to mount your camera behind and above your primary monitor. I use a large desk mount for this, but if you are often travelling or on a laptop, a smaller one works.

Base camera mount: Elgato Desk Clamp ($40)

Camera mount extension: Elgato Flex Arm ($40)

If you have a shotgun microphone, you should buy a mount for it too, placing it right behind your monitor and aiming the microphone towards you.

Desk-Mounted Boom Mic Arm ($72 with Amazon coupon)

Microphone Shock Mount ($13)

V. More accessories

You will need to connect your camera to the USB capture card, which then plugs into your computer:

4K Micro HDMI -> HDMI Cable ($10)

You will need to connect your microphone to your camera to include it in your HDMI stream, which also ensures it’s perfectly in sync with your video:

3.5mm Male to XLR Female Cable ($9)

If you want to use the microphone without the camera, you’ll want this cable

USB Male to XLR Female Cable ($13)

If you want to use your camera for outdoor filming, you’ll also want:

SD card ($12-$156)

SD Card Reader ($13) as well

If you don’t have a good lighting setup, you may want to purchase some lights as well:

Elgato Key Light – 2800 Lumens ($180)

VI. Configuration

You’ll have to change a few settings on the camera to have a good streaming experience.

It’s a popular camera, so if you don’t know how to do something, just search on Youtube. I found this video which helped me with a few settings such as: set the overheating threshold to high, add ‘USB streaming’ mode to the quick menu if you are not using a capture card, and turn the steady shot option off.

The most important item in this list by far is the camera – the microphone and lighting isn’t nearly as crucial for a good setup. Special thanks to Cory, CEO of Spellbrush, for helping me with all of this myself!

How to Twitter Successfully

Twitter (aka X) is one of the best social networks in the world.

It’s among the best places to make friends, find a job, find co-founders or investors, attend events from, date from, learn from, and now even make some cash on the side from. This is still true as of late 2023.

Despite this, many people haven’t given it a serious try and are missing out, likely because it takes time to learn how to make your Twitter experience great. In order to help remedy this, this post will cover:

  1. Twitter features you should use
  2. How Twitter accounts grow
  3. Social tips for success

Twitter Features You Should Use

I. Twitter Blue

Twitter blue is generally worth it. Although the value it adds in some areas is subtle, if it helps you out even a bit socially, it will easily be worth $8.

An example of this is if it encourages someone to respond to you, check out your profile, or read your DM, when perhaps they otherwise wouldn’t have. Twitter blue is purported to give algorithmic boosts, making you more likely to appear in the For You feed and causing you to appear higher in responses to parent tweets, but I don’t have explicit data to support this.

If your account has a large amount of impressions (5M+/month currently) you will make money from Twitter blue, so this should be a no-brainer. Based on the numbers I have, you should get a CPM of $0.01, although some accounts get higher rates. If you have 5M impressions per month, this should make you $50/month. Some accounts that I surveyed have a CPM of up to 5 times this such as Austen Allred. This may be due to having a much higher-value audience from an advertising perspective (many founders, investors, etc), or due to other unspecified favoritism.

I looked at the accounts I consider the highest-value, and around 50% have Twitter blue, so it’s a good signal that you’re a strongly above-average account.

II. Lists

source, well worth reading this thread!

You should probably be using the lists feature of twitter. Lists are a collection of accounts which constitute a separate feed which you can browse. You can share lists with others, or keep them private. Twitter lists also have no advertisements on the mobile app as of writing this.

Lists can be useful to segment Twitter by topic – maybe you want to add anyone who is an investor to a list in case you want to gauge investor sentiment only, or maybe you want to make a list of AI researchers only to see what papers are currently buzzing.

I personally use a ‘high priority’ list with ~100 accounts on it, allowing me to check this list in full daily with only a few minutes of time. This makes sure I don’t miss anything from the accounts that I think are the highest value. A subset of this list of people on my links page.

Although my lists are not public, some others are! Here are a few examples compiled by Lama:

III. DMs

Twitter has a DM feature. You’re probably under-using it.

You can just talk to people. It’s okay if they are a famous researcher, a CEO, or even a billionaire. Many of them not only have their DMs open, but will check them. The worst that may happen is you don’t get a response.

This isn’t to say you should be spamming people – you should definitely focus on starting a conversation when you think it will provide value to both participants. But Twitter is simply a network of humans, and humans love to socialize and make friends and help each other out, and it’s important not to forget this.

A cold DM on twitter from someone who you have ‘seen around’ is significantly less cold than a cold email, where you see nothing but an email address and name. If you are, for example, looking for a job at a company, you may want to look at who is hiring for that company on Twitter and ask them how you can improve your chances.

When I went to San Francisco for the first time, I didn’t know anyone there. I had zero friends. But what I did have was an anonymous Twitter account with 400 followers! I sent 6 cold DMs to some people who seemed cool and 4 of them agreed to hang out with me (one non-response, one busy). I had a great time with all 4, and we still chat on and off to this day.

I know a lot of people who have dated off of twitter, and many others who have met their wife or husband from Twitter. I haven’t done this myself, but it probably beats most dating apps.

Make sure your Twitter DMs are open (not verified-only, explicitly check this setting as for some users it was modified!) unless you have a good reason to close them.

IV. Muting & Blocking

You can mute keywords of things you don’t want to see. This was useful to many users during the NFT bubble, and in general can be a good way to keep politics or outrage-bait out of your feed.

You can also mute or block users. Muting a user ensures that you don’t see what they say, while blocking a user also ensures that they cannot respond to your tweets. Blocks can be considered rude, so muting may be a better idea in many cases.

Some users strongly advocate for the liberal usage of mutes and occasional blocks, although if you aren’t political and don’t engage with trolls (which is correct!), your need for them should be minimal unless you’re otherwise excessively controversial and/or popular.

Your twitter account is yours and your time here is likely limited, so make sure you’re enjoying yourself rather than spending your nights arguing with strangers.

V. Likes

Liking a tweet makes the twitter algorithm more likely to show you tweets similar to it. I don’t use the For You feature frequently, so my personal usage is a little different. I sometimes use likes in a manner almost close to read receipts: the cost of clicking like is very low, it’s nice to notify people that I have read their post, and it also means that my likes (which are public) are not particularly indicative of what I actually like, so it creates plausible deniability should someone point out that I ‘liked’ a tweet which goes against a given narrative.

As of late 2023, Twitter has an integrated bookmarks feature as well, although I don’t use it myself.

VI. Aggressively Curating Your Feed

If you see tweets from someone you don’t like, either unfollow them, mute them, or block them.

I generally unfollow accounts which are excessively political, and my Twitter experience is vastly improved as a result. Whenever something outrageous is happening that is covering the headlines (e.g. every day of ‘US politics’), I often don’t even see a single tweet about it. If it is something that actually matters and affects me, it’s likely someone I follow will bring it up.

Experiment with using the Following feed rather than the For You feed. I find my For You feed to be mediocre at best, and an easy way to waste time without getting much value, although some others find more value in theirs.

VII. Advanced search

Twitter has an advanced search feature which lets you search by account, engagement, date range, included words, excluded words, and more.

It is not the default search or accessible via the app, so many people do not know about it. You can use it by visiting this page: https://twitter.com/search-advanced

How Twitter Accounts Grow: 0 To 1,000 Followers

I. Foreword

Being popular on Twitter is probably not what you want.

You probably want something that correlates with it, like reputation, influence, friends, or money. You can make great progress on these metrics without having an absurdly high follower count. If you think do in fact desire true fame, my suggested reading for you is Reasoning Not to Become Famous by Tim Ferris, or the replies to any tweet Elon makes.

Which is more valuable, a twitter account with 100,000 everyday followers, or a twitter account with 1,000 followers, entirely comprised of CEOs, famous journalists, billionaires, and heads of state? I’d take the latter any day myself.

This may be an extreme example, but it’s true that higher-quality conversation is harder to find in the replies to larger accounts. Elon Musk may be an interesting person, but the average reply to his tweets is anything but that. I personally find the sweet spot of good conversations to occur with accounts in the 2K-15K range, but your mileage may vary.

Starting from zero followers sucks. Even if you post something good, it may go entirely unnoticed. Here are some tips to help you out.

II. Put an unreasonable amount of effort into your content.

This is the most important tip here, and that is why it’s first. The Internet is filled with content, and if yours is significantly better than average, it’ll help your odds tremendously.

If you’re summarizing a research paper, don’t just paste it into ChatGPT and tweet whatever comes out. Go over sections of it yourself, help explain it as clearly as possible, add or even hand-annotate and crop images yourself, and so on. A good example of an account that quickly grew from 0 -> 70K in a matter of months with this strategy is AI Pub.

If you have years of interesting experience in a field, you may just be able to tweet stream of consciousness thoughts and takes on things successfully, in which case the above doesn’t exactly apply: the unreasonable amount of effort that you put in was applied elsewhere (e.g. in your career), and you’re just translating your knowledge from there to Twitter. In general long posts are not a great idea, and should be separated into threads, although Andrej’s tweets are particularly high-quality, so I included him as an example.

III. Respond frequently, early, and with high-quality content

When someone popular tweets about something you know a lot about, respond to it with something useful (or funny). If you do this shortly after the parent tweet was made, there’s a good chance you will appear near the top of the responses, enabling you to piggyback off of the popularity of the original poster.

One of the best things about this is that people will notice. If you give high-quality responses, even accounts with 6 figures of followers will read them and notice. That’s all it takes to talk with the main characters of society, is your desire to post a response to them on Twitter.

IV. Source followers from external locations

If you have other social media accounts, or even any friends, you can direct people to your twitter there. You could also purchase followers, but that isn’t something that I’ll cover in this post as it’s not the type of value my prospective reader is after.

Most growth is based on your current number of followers (e.g., you should expect to gain a given percentage of followers per month), so it can take high-quality accounts months to go from 0 to 1,000 followers. Don’t give up and stick with it, and you’ll make it eventually!

How Twitter Accounts Grow: 1,000 To 100,000 Followers

After you have a few thousand followers, you’re at the point to where your tweets have a large initial seed userbase. This is great, because now if you post something with the propensity for virality, it has a much better chance of getting thousands of likes.

To help demonstrate how social media growth generally works, I’ve made a few charts of my Twitter metrics from a period where I was having fun growing my account.

Aggregate followers over time

This first graph is my followers over time. It is going up, and the slope is mostly increasing. That’s good.

To make it more useful, now we’ll adjust for the amount of followers that I had in each month, to show the relative growth instead of absolute.

Despite the number of followers increasing more and more over time, the percentage of followers that I gain in a month is surprisingly constant. My month-on-month growth was around 24% on average.

If you are familiar with the power of compounding, this should strike you as a very impressive metric, and is the exact type of thing venture capitalists look for in the revenue or usage metric of companies. A 24% MoM growth rate would amount to 1,300% per year or 40,300,000% across five years.

But, in some months I tweeted more, and in other months I tweeted less. Let’s account for that as well.

This graph is answering the question “for each tweet that I made, by what percentage did it cause my account to grow, on average, per month?”. Although it’s a bit messy, it is still surprisingly consistant, and its data has the lowest standard deviation out of all three graphs.

Thus, numerically speaking, to grow your twitter account:

  1. Tweet frequently
  2. Tweet consistently
  3. Do this for a long time, ideally for years

The best example of someone that has done this well, but for Youtube instead of Twitter, is MrBeast. He consistently made videos for years, getting very few views, but kept at it and kept improving until the subtle 10%/month gains compounded, and now 2% of the Earth’s population watches every video he releases.

With that said, none of this will matter if your tweets are low-quality. The above guidelines assume both that there’s some value in your content, but also that your goal is to maximize follower count. This isn’t the same as my personal goal, so I usually don’t tweet more than once a day, if that.

Okay, but what do I actually post?

Well, that depends on what kind of followers you want. You can become popular by posting 4chan memes, but if your goal is to network and get a job, this probably won’t help you very much. You could also become popular by posting research summaries of arxiv papers, but if your goal is to hang out with the boys and joke around, this might not hit the spot.

Broadly though, you should decide who you want to surround yourself by (you will become more similar to them, so be careful!), and what type of value you will provide in order to achieve this.

Most social media accounts can be mapped into a category based on the type of value they provide. Broadly, those categories are:

  1. Funny
  2. Interesting
  3. Useful
  4. Sexy
  5. Entertaining

These are rough categories, but if you think of some of your favorite twitter accounts, you should be able to map them onto one or more of the above categories.

The next section will go over more explicit advice that might help you to have a good time on Twitter.

Social Tips for Success

I. Be positive and constructive

The most important tip I have is to be positive and constructive. You can get engagement with dunks, but the followers and network you’ll end up with won’t be pretty. I’d avoid the political areas of Twitter at all costs.

II. Err towards saying things rather than being shy

This is hard for some people, but exposure therapy is the best way to fix it. Never be scared to tweet something because you have a lot of followers, or overly important followers, or anything like that, as long as it’s something you actually want to say. This is good advice for life in general. Trying things is good, and not trying things is bad.

III. Optimize your content for twitter

Linking to a 30 minute youtube video will generally get very low engagement, but specifically cropping out the best 30 seconds and adding a quick summary, quote, or thread will do much better.

IV. Don’t tweet walls of text

Both spacing out your tweets and tweeting with images are usually good ideas. If the first one or two sentences of your tweet are not interesting, few people will finish reading it. Twitter isn’t yet a good medium for long-form content, so I would usually not go above the 280 character limit, even with Twitter blue. I strongly advise reading Scott Alexander’s writing advice too, even if it wasn’t made for Twitter.

V. Make your own images

There’s a lot of value in making custom images. Most of my best performing tweets contained images that myself or someone else made, and which took as long as 10-30 minutes to make.

VI. Pseudonymity is cool but optional

Having a pseudonymous account can be very advantageous. A lot of people are scared to tweet their true thoughts publicly in a permanent form with their face and name directly adjacent to them. This is understandable and there’s nothing wrong with that. Even if you don’t have your name and face on your account, you can still make friends, meetup with people, and even network professionally or get a job, as long as you’re willing to share more details with individuals. A great example of someone who has managed this well is roon.

VII. Cold DM people more

You should cold DM people more. It’s a great way to get a job, make a friend, find a partner, and much more. This has explicitly been included twice on this page for a reason!

VIII. Only follow people you want to become more similar to

Only follow someone if you want to become more similar to them in some way, at least with respect to the content that they tweet about. Following someone gives them a limited type of write access to your brain, which for powerusers may be reinforced multiple times a day over the course of many years. This will significantly alter the type of person that you become, so use this super power wisely.

IX. Optimize for virality only at the cost of your soul

I would advise not purely seeking virality, even if you manage to avoid politics. Our best selves are probably not consistent with the versions of ourselves that maximize engagement online. I’ll leave you with the below quote from Dario Amodei, CEO of Anthropic:

That’s all I have here for now! If you have feedback to add, please add it to my tweet for this post or send a DM. If you made it this far you may also like some other posts on this site.

FAQ / Addendum

Hasn’t Twitter gotten worse with Elon?

I don’t personally find that the amount of value I get from it has gone up or down by much since the acquisition. Although some updates have been negative, I’m glad new things are at least being tried. I also exist in an area (AI, startups, tech, etc) which likely uses and enjoys Twitter more than average.

Regardless, you don’t have to be a fan of the CEO or owners of a company to use a product from them, and Twitter is no exception.

What are some other resources similar to this?

How did you calculate CPM?

I compiled self-reported impressions and payments from some accounts. I do not have a good sample size, but the data I have is available here:

AccountImpressionsPaymentCPM
https://twitter.com/nearcyan7,100,000?$454.62$0.064
https://twitter.com/Austen67,500,000$3,337.51$0.049
https://twitter.com/nickfloats31,300,000$1,253.88$0.040
https://twitter.com/FlyeKitesJay6,200,000$66.30$0.011
https://twitter.com/markgadala61,100,000$667.63$0.011
https://twitter.com/Danizeh60,000,000$650.00$0.011
https://twitter.com/concept_central/21,400,000$120.65$0.006
https://twitter.com/dom_lucre/472,000,000$2,402.16$0.005

Leveraged and alternative ETFs: Investing with higher risk tolerance and significantly greater potential upsides

I often end up talking about finance a lot, and in doing so often mention investing strategies and asset classes that many regular retail investors aren’t aware of. Although the world of financial derivatives is vast and unknown to most, I wanted to make a brief post about some simple products which I think should have more publicity, primarily that of leveraged ETFs. This post is a brief introduction to some investing strategies that some retail investors choose to use for higher risk tolerance and significantly greater potential performance. This post is not investment advice, in case I need to actually say that. I should also add that suggesting holding leveraged ETFs for longer periods is a relatively controversial view within the wonderful world of dollarmancy; nonetheless I present my own views here honestly should anyone wish to know them.

stonks
this could be you!

Cash is not your friend

At the lowest level of risk tolerance, many choose to simply keep their savings in cash. This is bad when done for longer periods. It is often pointed out that you will slowly lose money to inflation over time (whether that is the 2% inflation rate that the FOMC targets per year, the ~7% rate of 2021, or perhaps much more..), which although true, is not nearly as large of a loss as the opportunity cost incurred by investing in nothing. Many will provide APY estimates for investing in common market indexes between 6% and 9%, but examining as much as the last few decades (or even just the last decade) will show significantly greater numbers. $SPY has returned over 10% per year since its inception 29 years ago, and around 16.5% per year for the last decade (the last 3 years are even more impressive at 26% each on average!). I do not attempt to claim these are indicative of future results, or that we should be promising anyone these numbers, but it does seem to be unfair to weigh our expected market growth by including past decades that go so far back that we lacked not only much of our modern monetary policy knowledge, but also inventions as basic as the Internet itself.

If casual returns of 10.5% per year were not enough to motivate oneself, I often like converting these APYs to the period of a decade – in which case 10.5% corresponds to a 171% gain (1.105^10), 16.5% APY to a 360% gain, and 26% APY to a >900% gain. We could, of course, make these numbers even more grandiose by telling someone what returns they may expect by holding an investment for 20 or 50 years, but I find a decade to be a relative sweet spot, perhaps because people have an easier time imagining themselves a decade in the future rather than several.

I have heard many reasons for why people choose not to invest in ETFs (or anything similar such as individual stocks), from the reasonable “I am purchasing a house in a few months and now is not the time to take on any risk”, to the questionable “I am waiting for things to cool down a bit and I am a bit worried about some things in the near future”, to the absurd “I do not trust wall street or bankers, sorry” (and indeed, much can be said about how poorly we educate our citizens in the US about basic personal finance, which unfortunately involves much more than just basic investing). I am not going to spend many words attempting to convince someone that holding cash long-term (a year or more) is sub-optimal, because it seems obvious enough to me that it’s considered outside the scope of this post.

What margin is and isn’t

Most young professionals are now fully aware of what index funds are, and often have some simple strategies for investing in them. While it’s not my job to decide the risk tolerance of others, I do think it’s nice to at least be aware of some options that can generate significantly higher long-term returns than these traditional index ETFs. This is not investment advice, and regardless of if it was, I would not want to be responsible for someone else’s choices should things turn south.

The primary product I’d like to mention is that of leveraged ETFs. Many will initially recoil upon hearing the term ‘leverage’ mentioned in the context of personal finance, because they know that it’s scary and can be involved in situations where someone loses their entire principle (that is, 100% of their portfolio). It’s for this reason that I want to start with mentioning the difference between buying stocks on margin and purchasing a product which itself uses margin.

Buying stocks on margin is generally considered to be risky, because you are buying more than you can afford with your own money, effectively taking a loan from your broker in order to afford additional shares. Generally leverage of up to 4x is attainable with popular large-cap stocks on most US brokers, although there’s many exceptions to this. Although buying stocks on margin is not something I would generally suggest for many reasons, it does have a lot of uses, and it can be much less intimidating and dangerous than many may guess. Tools to analyze, manage, and properly limit one’s risk to a comfortable level are readily available, and rates for margin loans can be as low as 1% or under (IBKR is generally the golden standard for the lowest margin rates for regular retail investors, but some other platforms do offer better interfaces, tools, or additional products, and will also be able to negotiate rates with you should you have sufficient capital).

The obvious downside to margin is that you can lose much more of your investment. Theoretically, if you bought a stock with 4X leverage and it then declines by 25%, you would find yourself broke. In practice, you will get liquidated by your broker before this happens, unless the 25% decline happens instantaneously and they do not have enough time to sell your securities on your behalf (If you have heard the term margin call before, that is what happens when you do not have enough capital to maintain your leverage, generally after whatever you own performs very poorly. You can either deposit more money to get back to your maintenance margin, sell some of the products you own via leverage, or let your counterparty liquidate them for you). I am not going to get into the different types of margin or ideal scenarios for using it (of which there are many – remember, this is a loan with an interest rate of only 1%!) in this post, but rather have included this information to help it contrast with what a leveraged ETF is.

Leveraged ETFs

A leveraged ETF is not the same as buying stocks on margin. It is similar in that it is a higher-risk investment that easily allows one to lose or gain much more than usual, but it is different in that you are not taking out a loan explicitly nor implicitly, are not in debt, and therefore cannot be margin called, liquidated, or otherwise lose your shares via any means except via deciding to sell them yourself (this doesn’t mean they can’t still decrease in value by an arbitrary amount of anything less than 100%, however).

A leveraged ETF functions similarly to a regular ETF – it is a security that you can purchase, in which the work of managing your portfolio is abstracted away from you, and instead done by the issuer of the ETF. Instead of buying shares in 500 companies and managing their proportions yourself, you can simply purchase a share of $SPY and forget about it. In exchange for this convenience, you are charged a fee of 0.094% per year (this is often listed by brokers and compiled by ETF websites, but the original source is in the prospectus for the given security). The goal of an ETF is to track its underlying index – if the S&P 500 index is down by 1% in a given day, $SPY should be down close to that amount as well. A leveraged ETF attempts to perform the same function, however it introduces a linear multiplier which multiplies the intended gains and losses. In the US you will generally only find products that offer 2x or 3x leverage due to SEC regulations (3x products are often grandfathered in, as a 2020 update from the SEC suggests a general cap of 200% leverage via derivatives being allowed), although this introduces much more than enough additional risk and volatility for most investors’ appetites (should one want more leverage, they can create additional artificial leverage through the use of options, but that is also outside the scope of this post. Also, gambling is bad, Just Say Neigh!)

Leveraged ETFs are re-balanced daily, and thus intend only to match the performance of their underlying index (multiplied by 2 or 3) for a given day. If the S&P 500 index goes up 1.5% in a day, then a 2X leveraged ETF for it should return close to 3% that day. Due to their targets being daily, some investors often misinterpret this as being equivalent to matching returns on longer periods, although this is not the case. This has been misunderstood enough that the SEC has an alert attempting to inform investors of this, providing some historical examples of leveraged ETFs declining in value during longer periods, during which the underlying index performed positively. This is generally referred to as ‘volatility drag’, and is one of the largest reasons for which many discourage investors from purchasing these products. Much has been written about it, so I will just offer a very short summary: during periods of volatility, leveraged ETFs will perform worse than one would expect at first glance. To give a simple example as to why, imagine that portfolio A returns 5% on day one and then loses 5% on day two. If you started with $100, you will end up with $99.75 ($100 * 1.05 * 0.95). If portfolio B multiplied these daily fluctuations by 3X and returned 15% on day one and -15% on day two, $100 would turn into $97.75 ($100 * 1.15 * 0.85). As you can imagine, if we iterated over these scenarios many times, portfolio B would start to perform terribly in comparison to the portfolio with less leverage.

Volatility drag, aptly-named, is bad during periods of volatility, but it’s particularly bad when there’s not enough underlying momentum in the upward direction to counteract it during longer periods. During a market that is performing even moderately well, generally the greater returns provided by leveraged products don’t just return more than is lost due to volatility drag, but return so much more that being fearful of the concept can be actively harmful (this is likely a controversial opinion in many areas, for what it’s worth – but many people become scared of an investment that could feasibly return 1,000% over a period because of a potential loss of 10% or 50%, even if it’s clearly a very high expected value. In some cases this may be rational due to the diminishing returns of utility provided by additional capital (money may buy a little happiness, but this caps out pretty quickly, and having no money is definitely much worse than having just a little!), but it is well-known that humans are far too risk-averse as a general principle regardless).

To provide some examples, I will mention some leveraged ETFs alongside the returns that they have provided historically. As usual, past performance is not an indication of future results!

$SPUU, a 2X-daily-leveraged ETF that tracks the S&P 500 index, has returned an average of 32% annually for the last 5 years, and 27% annually since inception. $SPXL, a 3X-daily-leveraged ETF that also tracks the S&P 500, has returned an average of 41% annually for the last ten years. Those of you used to performing basic calculations on compounding annual rates will quickly realize how absolutely insane these numbers are – 41% returns compounding for a decade comes out to a return of +3,000%! This is something that is possible, and that many investors have actually attained, providing they didn’t sell during draw-downs (this is not the same as it being guaranteed, or even probable, however).

If past performance is not a promise of future performance, then why is it being mentioned so saliently here? Because although strong performance is not guaranteed, this helps to illustrate the potential of what happens with leveraged ETFs when things go really well, which we can reasonably say has been the case since 2010 to 2022. Because things are not guaranteed to go well, putting 100% of your net worth into these leveraged products is reckless and is very likely a bad idea. However, just as some people like to have hedges just in case things go south, I think it’s important to have some minor positions in place just in case the opposite occurs: If we get lucky and the next 10 years go as well as the last, it is quite possible to attain a 20x, 30x, or greater return on your investment. If you get unlucky, you may lose some or most of your investment, but no more than 100% of it, so the risk to reward is very strongly in your favor (yes, the math is much more complicated than this, but the result holds in more nuanced conditions regardless). In the next section I will go over a few basic common questions about leveraged ETFs, as well as mentioning more of the negatives.

Leveraged ETFs exist for most popular stock indexes, including sector indexes. For example, $SOXL is a 3X-leveraged ETF based upon the ICE Semiconductor Index, which primarily consists of companies related to semiconductor manufacturing. As it is my personal opinion that we are going to tile the world several times over with semiconductors (or something equivalent) in the coming decades, this is a product that I’m a fan of personally, even if it is very high-risk. For some listings of leveraged ETF products, check out out these pages from Direxion and Proshares

Responses to common concerns about leveraged ETFs

Aren’t leveraged ETFs not intended to be held for longer than a day?

This is mentioned in many locations, but it functions primarily for the purposes of legal liability and investor protection. There is nothing wrong with holding these products for longer periods, as long as one is properly educated about them. This is the type of warning where those that it does not apply to will know they can ignore it. There are other similarly-accessible products that are much worse ideas to hold for longer durations, for example inverse-leveraged ETFs, which return the opposite of what the underlying index returns, and thus trend towards zero over the long-run (for an example, $SPXS has returned -47.22% since inception, which leads to over a 99% loss after a decade. If you’re curious why inverse ETFs exists, they are primarily for short-term speculation and various types of hedging).

Aren’t leveraged ETFs subject to volatility drag, and thus a bad idea to hold long-term?

As mentioned above, volatility drag is an important thing to be educated about and aware of. However, if markets actually perform well, the potential gains from leveraged ETFs significantly outweigh (often by more than an order of magnitude) losses due to volatility drag. Regardless, it is worth noting that as many leveraged ETFs are recent financial products, there is an inherent cherry-picking present in the data used to show how well they perform, as the previous 5-20 years have been favorable financially for most US sectors.

Don’t leveraged ETFs have much higher management fees than most normal ETFs?

This is true, and is also something to note. As with the above two examples, $SPUU’s gross expense ratio is 0.88%, and $SPXL’s is 1.03%. Similarly to volatility drag, while it’s important to be aware of these expenses as they do add up and eat into long-term profits, if the market performs well, you will make so much that you will not even notice it.

I don’t want to get margin called, gamble with money that is not mine, or be in debt

Luckily none of these things occur when purchasing leveraged ETFs. You can still lose almost all of your money, but you cannot go into debt or have your shares taken away from you (unless you are engaged in other things that may cause this).

Leveraged ETFs have draw-downs that are far too high for the risk tolerance of every day people

I would say this is completely true. If we take a fund like $SPXL and look at what happened during the covid crash, it crashed from $76 to $18 in a single month, or a decline of around 77%. Apart from this being bad financially, drawdowns this large often cause significant emotional distress to investors and can easily cause them to make poor choices and panic-sell at market bottoms. While $SPXL may have returned back to $76 in less than a year (and then somehow doubled in the year after that..), this will obviously not always be the case. It’s quite possible for drawdowns in some leveraged ETFs to reach 90% or more, even if very rare.

This is gambling

All investing is gambling, mathematically speaking. The absence of investing is also gambling due to opportunity cost – if you hold USD, you are literally betting for it and the US to do well! While it’s true that this is more like gambling than other financial products in the views of many, it should not be compared to acts such as buying a lottery ticket or going to a casino, where there is a known large house edge against you, with the objects in question having been specifically constructed in order to gain the upper hand over you.

Markets exists everywhere and will not go away any time soon, so there is no option of ‘not playing’ the game, as unfortunate as that may be for some of us. The only question is what one’s risk tolerance and personal choices are, not whether they exist or not, because they are forced into existence by our environment. While it may be easy to lose a lot of money on leveraged ETFs, it is nowhere near as bad as buying short-term out-of-the-money options, binary options, 100x leveraged cryptocurrency swaps, 250x forex trades, writing uncovered cryptocurrency options, and many other ‘fun’ products that exist and are often traded by young males addicted to gambling.

How much of my money should I invest into leveraged ETFs?

I have no clue; the right answer for you, dear reader, could very well be 0%, 100% or anywhere in between, but I am not the one that can decide for you. I can say that it is worth your time to learn a lot about how personal finance works however, regardless of your risk tolerance or intentions.

Something something trading leveraged-ETFs or other things

Although I am not in the business of telling people what to do financially, I do enjoy telling people things I think that they should not do, and one of those is ‘trading’. The short version of my advice on this matter is that you should be buying and selling things as infrequently as possible, and you should avoid things like ‘day trading’ like the plague. If you find yourself constantly checking prices, you are likely over-leveraged. I have watched too many bad things happen to too many amazing people, many of them very smart, and most of them young males, and I want to do what I can to cause gambling addictions and casual day-trading to happen less. The humor of places like r/wallstreetbets may be quixotically funny at times and comically sardonic at others, but behind all of the fun people are having with memes about cryptocurrencies and options on Reddit and Twitter, lay thousands of people who have lost their life savings, many of which who end up taking their own lives or losing decades of accumulated capital. Markets are not a game, and if they find a way to eat you alive, they will, as they have become exceedingly efficient at it in the recent few decades.

Further Reading

Any thoughts, corrections, suggestions? Feel free to say hi on Twitter or provide Anonymous Feedback

Pascalian Longevity: Why not?

Scott Alexander of SlateStarCodex / AstralCodexTen recently wrote Pascalian Medicine, in which he looks at various substances purported to improve covid outcomes, but which have relatively low amounts of evidence in their favor, likening administration of all of them to patients to a Pascal’s wager-type argument: if there is a small probability of a potential treatment helping with covid, and if it’s also very unlikely that this treatment is harmful, should we just give it to the patient regardless of if the quality of evidence is low and uncertain, as it would clearly have a positive expected outcome regardless?

The naive answer to this could simply be to attempt to calculate an expected value (note: I use the term expected value often here, but in some cases the terms hazard ratio, relative risk, or odds ratio would be more appropriate) for each treatment, and administer it if it’s positive. But there could be some unintended consequences of using this methodology over the entire set of potential treatments: we could end up suggesting treatments of 10 or 100+ pills for conditions, and apart from something just feeling off about this, it could magnify potential drug interactions, some treatments could oppose others directly, the financial cost could start to become prohibitive, and it could decrease patient confidence and have many other undesirable second-order effects.

Pascalian Longevity

There are many counter-arguments presented to the above concept which become less salient when the goal is changed from ‘find drug treatments to prescribe to all covid patients’ to ‘find personal health interventions that increase your own lifespan/longevity’.

I am fortunate enough that I am able to evaluate potential longevity interventions myself, pay for them myself, administer them myself, and review their potential effects on me myself. I might not do a perfect job of this – research is difficult, time-consuming, and lacking in rigor and quantity, and finding appropriate longevity biomarkers to quantitatively asses the effects of interventions is also difficult. But uncertainty is a given here, and that is why we incorporate it into our frameworks when deciding if something is worth doing or not by calculating an expected value. Furthermore, any harm that I may accidentally incur will only be done to myself, reducing the ethical qualms of this framework to near-zero (I would strongly oppose arguments that I should not have the right to take drugs which I think may significantly improve my own health, although some may disagree here).

My modus operandi with respect to longevity may have many uncertainties in its output, but still operates with a very strong (in my opinion) positive expected value: If a substance significantly and consistently increases the lifespan of organisms similar to humans (ideally in humans), and is also very safe in humans, then it is something that I want to take

This is how I operate personally with longevity, and it does result in me taking quite a few things (currently I’m at around 15). I do still try to minimize what I take as a meta-principle (for example, setting a minimum threshold of expected value that a substance must provide to warrant inclusion, rather than simply accepting any positive expected value) for a few reasons: firstly, to reduce potential drug interactions (which we do attempt to asses on a per-substance basis, rather than account for as an unknown, but unknowns are unfortunately a very large component of messing with biology regardless). Secondly, to keep my costs relatively sane, although I am not too worried about this as there are few ways to spend money more effectively than on trying to improve your health. Thirdly, to reduce the occurrence of interventions that may have the same or opposing mechanisms of action (taking two things with the same mechanism of action may be okay, but sometimes dose-response curves are less favorable, and taking >~2x of something will result in diminished or even negative returns). Lastly, to minimize potential secondary side-effects that could be cumulative over large classes of substances (for example, effects on the liver).

I don’t intend to promote any specific substances or interventions here as I don’t give medical advice, nor do I want anything specific to be the focus of this post, but I do want to remind us that just as we can calculate expected values in a utilitarian fashion and get effective altruism as a result, we can do the same for longevity interventions and get a very strong chance at notably increasing our lifespan/healthspan as a result. I do have a list of some of what I take here, but it is definitely not intended to promote anything specific to others.

Why Not?: Potential counter-arguments

Algernon’s Law

Algernon’s Law is sometimes brought up, suggesting that evolution has already put a lot of effort into optimizing our body, and thus we are unlikely to find improvements easily. But, as Gwern notes in the above link, there’s at least three potential ways around this reasoning: interventions may be complex (and/or too far away in the evolutionary plane) and could not have easily been found, they may be minor or only work in some individuals, or they may have a large trade-off involved and cause harm to reproductive fitness.

Although some areas of future longevity treatments may fall under exception one and be complex enough that evolution could not have found them, I would suggest that the majority of today’s potential treatments fall under exception three: evolution optimizes for reproductive fitness, not for longevity, and for this reason there are many interventions which will improve our longevity that it has not given to us already (this is part of why I am more optimistic about longevity interventions than I am about intelligence interventions/nootropics).

For an extreme example of this, it has been noted that castrated males often live longer, and that this is obviously something evolution would not be very interested in exploring. Although this has been found with median lifespan in male mice (maybe in females too?), there is also purported historical data on Korean eunuchs suggesting that they may have lived a full 14-19 years longer (there are definitely potential confounding variables and/or bad data here, but we don’t have RCTs on this in humans for obvious reasons..), and a more recent study in sheep that is also highly relevant: Castration delays epigenetic aging and feminizes DNA methylation at androgen-regulated loci, where epigenetic aging clocks that look at DNA methylation are used in castrated sheep. There are other traits that seem to improve longevity as well, for example decreased height. It seems quite plausible that there are a lot of trade-offs that optimize for strong reproductive fitness early in the lifespan of organisms, which end up costing the organism dearly in terms of longevity. These trade-offs may be involved in many areas such as testosterone, estrogen, growth hormone, IGF-1, caloric restriction, mtor activation, and many others.

Large error in estimating unknown risks

One other counter-argument here is often along the lines of “you are messing with things you don’t understand, and you could be hurting yourself but be unaware of this; the damage may also be difficult to notice, or perhaps only become noticeable at a much later time”

It is true that our understanding of biology is lacking, and therefore also that we are operating in highly uncertain environments. I would be open to evidence that suggests reasoning for why we may be systemically underestimating the unknown risks of longevity interventions, but given how strong the potential upside is, these would have to be some pretty terrible mistakes that are being made. It is often noted how curing cancer may only extend human lifespan by a few years, whereas a longevity improvement of 5% for everyone would provide much more value (and is also much easier to find in my opinion). One could make an argument here that even if I was doing something that notably increased my risk of e.g. cancer, if the expected lifespan increase of this intervention was as much as 1-5%, this could still be a huge net positive for my health! I don’t take approaches that are this extreme regardless, and I try to keep the risk side of my risk/reward ratio low independently of the level of potential reward in attempt to account for this uncertainty. I am also not aware of many interventions that seem to have very high numbers in both the numerator and denominator here, although I am pretty certain that they do exist; I don’t currently take anything that I think has notably detrimental side-effects for the time being.

Is it fair to call this approach Pascallian?

The original nature of Pascal’s wager is that of extreme probabilities resulting in positive expected values, but the numbers that we are operating with are nowhere near as extreme as they could be. It is probably not a good idea to take 10,000 supplements, each of which have a 0.1% chance of extending your lifespan by a year for many reasons (similarly, if 10,000 people that claimed to be God all offered me immortality for a small fee, I would hope to decline all of their offers unless sufficient evidence was provided by one).

As I’m not arguing in favor of taking hundreds or thousands of supplements in the hopes that I strike gold with a few of them, it may be worth noting that ‘Pascallian Longevity’ would be a poor label for my strategy. Regardless, taking just 5-10 longevity interventions with a strong upside potential seems to be significantly more than almost everyone is doing already, so I still stand by my claim that there are many free lunches (free banquets, if you ask me) in this area, and I am very optimistic about the types of longevity interventions we’ll find in the coming decades.

Open to any corrections/comments on Twitter or any medium on my about page

Favorite Links

This page lists many of my favorite blog posts, organized by author. Much of my most-cherished knowledge is from blog posts or internet comments, so I hope to share some of that with others here. Last updated: Feb 17, 2024

Scott Alexander (Twitter): As the author behind SlateStarCodex (now AstralCodexTen) and many great LessWrong posts, Scott is among one of the best written content creators of the last decade. He writes about psychiatry, rationality, and meta-science. Here’s some writing of his that I love, with my favorites bolded:

Gwern Branwen (Twitter – currently private): Well-known for having quality deep dives in diverse areas such as statistics, technology, machine learning, genetics, psychology, and many others. Also often recognized as an amazingly aesthetic, verbose, and highly-usable website. Favorite posts:

  • About Gwern: About Gwern; who he is, what he has done, and links to other mediums
  • It Looks Like You’re Trying To Take Over The World: An eloquently-written and humorous short story about AI alignment and paperclipping, featuring our good friend Clippy alongside a multitude of entertaining references, both to Internet history and many arxiv machine learning papers
  • Generating Anime Faces: An overview of GANs in machine learning, with focus on Stylegan2 and anime art generation including ThisWaifuDoesNotExist and its follow-up ThisAnimeDoesNotExist, both trained from a large Danbooru dataset
  • Death Note Anonymity: Using information theory to quantify the magnitude of Light Yagami’s mistakes in Death Note (absolutely worth watching, even if you’re not into anime), offering insightful analysis and constructive criticism
  • The Scaling Hypothesis: Discussion of the scaling hypothesis in machine learning (essentially how much better models get with significantly more data+compute), with obligatory emphasis on GPT-2 and GPT-3
  • Melatonin: Detailed information on melatonin, a simple endogenous hormone that notably improves sleep in many individuals when supplemented just before bedtime
  • Nicotine: An analysis on the benefits of nicotine as a nootropic, with attention given to the fact that it is often incorrectly assumed to be a dangerous and addictive drug due to its inclusion in cigarettes and consequently significantly-confounded research claims
  • Modafinil: Discussion of modafinil, a prescription stimulant drug that appears to have a relatively favorable cost/benefit profile for productivity and alertness
  • Embryo Selection for Intelligence: A cost-benefit analysis of the marginal cost of IVF-based embryo selection for intelligence and other traits
  • Why Correlation Usually ≠ Causation: A meta-scientific discourse and analysis on the age-old adage that correlation does not imply causation
  • The Melancholy of Subculture Society: A brief analysis on the cultural effects of the Internet allowing niche subcultures to easily form
  • Newsletters: Links to Gwern’s past ~monthly newsletters
  • My Anime List: Gwern’s top-rated anime

Andrej Karpathy: (Twitter) A bright AI researcher who has spent time both at OpenAI and as the chief AI officer at Telsa. He has a popular Youtube channel with machiune learning content as well.

  • The Unreasonable Effectiveness of Recurrent Neural Networks: Back in 2015 Andrej trained a 10M-parameter RNN on some interesting text datasets like the source code for the Linux kernel and Shakespeare. Performance was surprisingly good!
  • Biohacking Lite: It’s always fun to read content from people from fields like computer science when they later deep dive into biology, often for their own personal health. This post has some high-SNR content on the basics of metabolism and energy in humans as well as some quantified-self demonstrations and simple dietary advice.

Scott Aaronson: A theoretical computer scientist with a focus on quantum computing and complexity theory. Although his posts on quantum computational complexity theory research go over my head, I’ve enjoyed some great content from him in other categories. Favorites:

Matt Levine (Twitter): An ex-Goldman Bloomberg opinion columnist with some wonderfully insightful and hilarious posts (offered as a free newsletter, generally ~4x a week) on the happenings in our modern yet often-insane financial world. Posts are generally centered around current events and are best read as they come out. Some examples:

Nintil (Twitter): A wonderful blog by Jose Luis Ricón with a focus on longevity, economics, and meta-science. Favorite posts:

Patrick Collison (Twitter): The CEO and co-founder of Stripe, often with focuses involving meta-science, individual and societal productivity, and economics

  • Fast: Examples of people quickly accomplishing ambitious things together
  • Questions: A short list of interesting questions
  • Advice: Advice, particularly for young and ambitious individuals
  • Book Recommendations: A well-sized list of suggested reading

Sam Altman (Twitter): The CEO of OpenAI and former president of Y Combinator, his posts often focus on startups, artificial intelligence, productivity, and science. Favorites:

  • How to be Successful: Thirteen thoughts on how to achieve long-term successful outcomes: learn a lot, compound yourself, work hard, and be ambitious
  • Productivity: Various productivity tips, such as ‘Picking the right thing to work on is the most important element of productivity and usually almost ignored. So think about it more!’
  • Advice for Ambitious 19 Year Olds: Advice for young and ambitious individuals, such as ‘The best people always seem to be building stuff and hanging around smart people’
  • How to Invest In Startups: Advice about being a good startup investor
  • Super successful companies: Notes some salient commonalities between many very successful companies
  • The Strength of Being Misunderstood: You should trade being short-term low-status for being long-term high-status

Paul Graham (Twitter): The founder of Y Combinator, with many posts focusing on startups, ideas and frameworks for everyday life, as well as advice and reflections for people that fit the founder/builder/nerd stereotype. Some favorites:

  • Do Things That Don’t Scale: An amazing tip on gaining initial traction and leverage by doing high-impact activities that won’t scale, but that will work effectively for the time being
  • What You Can’t Say: Reflections on that which exists outside of the Overton window
  • How to Make Wealth: An essay on effectively building wealth over time
  • Keep Your Identity Small: On why politics and religion yield such uniquely useless discussions due to excessive involvement with personal identity
  • Having Kids: Personal experiences and thoughts on having kids
  • It’s Charisma, Stupid: A 2004 essay arguing that charisma is the most important trait for elected politicians, using the US presidency as an example
  • What I worked on: A personal and emotional memoir on pg’s professional and personal history

Alexey Guzey (Twitter): Currently working on New Science, Alexey has some great blog posts with a focus on properly using the Internet for social leverage (reach out to people more, cold email people more, initiate conversations more, and create content more!), meta-science, productivity, biology, and more. Some favorites:

Melting Asphalt (Twitter): Written by Kevin Simler (along with Robin Hanson (Twitter), co-author of The Elephant in the Brain), Melting Asphalt has a wonderful collection of posts on evolutionary psychology, game theory, and novel and introspective takes on what makes us human. Favorites:

  • Neurons Gone Wild: A beautifully speculative post that suggests a recursively selfish model of biological neurons which enables selfish sub-agents and networks to co-exist in an evolutionary semi-competitive environment within our own minds. Probably my favorite post on this blog for several reasons. Also see Hallucinated Gods
  • Music in Human Evolution: A great book review of Why Do People Sing?: Music in Human Evolution by Joseph Jordania, involving predatory defense mechanisms, disposition of the dead, battle trances, and the audio-visual intimidation display
  • Crony Beliefs: On beliefs that stick around when they shouldn’t
  • Personality: The Body in Society:
    What is personality? ‘Nature and nurture work together to create a prototype, which then negotiates with the external world. The result is a strategy for getting along and getting ahead — a strategy we call “personality”, in other words, ‘Personality is a strategy for making the most of one’s particular lot in life.’ See also: part two and part three
  • Ads Don’t Work That Way: On ‘cultural imprinting’ and signaling in advertising
  • Doesn’t Matter, Warm Fuzzies: Discusses many interesting aspects of human ecology and society, with a focus on rituals, culture, confabulation, mimicry, and more
  • Social Status: Down the Rabbit Hole: On social status in humans, including an analysis of two proposed separate status systems: dominance/submission and prestige/admiration. See also: Social Status II: Cults and Loyalty
  • Border Stories: Borders are a necessary precondition for agency within a hostile ecosystem

Telescopic Turnip: Reads like type of cross-over between scott alexander and gwern, which means it’s good

Qualia Computing: With a subtitle of ‘revealing the computational properties of consciousness’, Qualia Computing is a great blog for anyone interested in the neurology, phenomenology, and interesting attempts at quantifications and explanations behind our own conscious experiences (qualia)

Patrick Mckenzie (Twitter): An entrepreneur and writer that lives in Japan and currently works at Stripe with a focus on startups and outreach, Patrick has many invaluable posts about finance, startups, marketing and professional communication, and highly-regarded SaaS and entrepreneurial advice. Favorite posts:

Nat Friedman (Twitter): Great personal website!

Fantastic Anachronism (Twitter): todo, see Recommended Reading

Applied Divinity Studies: todo

Peter Attia (Twitter): todo

Vitalik Buterin (Twitter): todo, see The bulldozer vs vetocracy political axis

Lesswrong: todo

Overcoming Bias: todo, ‘This is a blog on why we believe and do what we do, why we pretend otherwise, how we might do better, and what our descendants might do, if they don’t all die’, from Robin Hanson.

Aella (Twitter): Popular Posts, Becoming A Whorelord: The Overly Analytical Guide To Escorting, Handling Accusations In Communities, The Polyamory Post

Tim Ferris: 11 Reasons Not to Become Famous

Dynomight (Twitter): todo, see Better air is the easiest way not to die by The impact of air pollution on health is often significantly underrepresented, and working on improving the quality of air in your dwelling can result in a very high ROI for your health

Marginal Revolution: todo

Allulose: The Best Sugar Substitute

Allulose (sometimes D-psicose) is by-far one of the best ways to add sweetness to home-cooked meals in a healthy and low-calorie way. As an epimer of fructose, it has been steadily gaining popularity within the last few years, and not without good reason! Allulose is not only nearly calorie-free, but also decreases blood glucose levels with meals, and seems to have a wide range of potentially beneficial effects.This post is a short summary of why allulose is so appealing over sugar and other sugar substitutes.

70% as sweet; 100% as white and crystalline

Overview of Allulose

Allulose is found naturally in wheat, figs, raisins, maple syrup, and molasses, although in relatively trace amounts. It has around 10% the calories of traditional sucrose and can be manufactured from fructose. It’s around 70% as sweet as sucrose (regular sugar), but has a similar taste and feel, which is a large factor behind why it makes a great substitute (or partial substitute) for baking or dissolving into things. The taste of Allulose has a more natural and relaxing quality than some other sugar-replacement options such as xylitol and erythritol, which are both sugar alcohols, but generally have a ‘cooling effect’ (often likened to the aftertaste of consuming mint, which allulose conveniently lacks).

Allulose is also an actual sugar (not a sugar alcohol or other compound), and has similar browning properties to sucrose via the Maillard reaction. One downside to mention is that it does seem challenging to keep some styles of baked goods crunchy with allulose as the only sugar; while it definitely seems to be one of the best options for sweetening drinks, yogurts, ice creams, cakes, and so on, it may not be the best option for super-crunchy cookies (although can make great softer ones!). This seems to be due to allulose not crystallizing when it cools, its ability to hold more moisture, and that it is more soluble in liquids than sucrose; hence it being a great fit for drinks, sauces, and spongy baked goods.

Allulose was designated as GRAS by the FDA in 2019, so is still relatively new to the market compared to many other sugar substitutes, although has been gaining significant popularity for the short period that it has been available for general usage in foods. I’m sometimes now able to find allulose for sale in a supermarket or included in a sweet good (and it is also now being used in products such as Soylent), although its usage is still a small fraction to that of sugar and corn syrups. It can easily be purchased on Amazon for around $10 per lb (regular sugar is generally closer to $1-2 per lb, so it is quite a bit more expensive if you happen to use very large amounts of sugar).

What Sets Allulose Apart

Why might we want alternative sources of sweetness from sucrose to begin with? Although much has been said about the ways sugars are (in some cases) potentially harmful, it seems reasonable to posit that there are two qualities of a diet with high sugar content (remember, this means any typical western diet!) that are undesirable: firstly, the high caloric content of sugar, which makes over-eating significantly easier and therefore contributes to obesity, and secondly, the effects of sucrose on blood glucose levels and thus insulin resistance, which contributes to diabetes and metabolic syndrome.

As we would hope from an alternative to sucrose, allulose doesn’t cause an increase in blood sugar. The reason for this is that it is not absorbed and digested by the gastrointestinal tract, but rather processed by intestinal bacteria. For the most part this is a good thing, and is what enables allulose to both be low-calorie and to not be converted to glucose in the blood stream. The downside of this is that for some people, especially if consumed in large enough quantities, it can cause mildly discomforting side effects such as flatulence, subpar digestion, and abdominal discomfort. This is much more likely to occur if you, for example, eat an entire batch of allulose cookies by yourself (who would do such a thing..!?), rather than simply use it to sweeten a drink or a snack. While I haven’t experienced anything negative myself, everyone is certainly very different when it comes to food.

But, it gets much better than this! Allulose not only doesn’t increase your blood sugar, but actually decreases it! It does this by inhibiting alpha-glucosidase (along with several other similar enzymes), which is an enzyme that breaks down starches and disaccharides into glucose (i.e. causes carbohydrates to lead to blood glucose spikes). Other well-known inhibitors of alpha-glucosidase include acarbose, a popular and simple diabetic drug which significantly extends lifespan in mice and has the exact same potential side effect profile as large allulose doses (and in my opinion is probably very good for most people to be taking, perhaps extending human lifespan via the same mechanism of action as in mice), and sweet potatoes (source, another source). Thus, adding allulose to meals that contain carbohydrates will result in less of a blood glucose spike than if allulose had been excluded.

Comparison of blood glucose area under curve for small quantities of fructose vs allulose (source: figure 1)

There’s now quite a few studies showing this in humans (and dogs and mice!), with allulose consistently attenuating the postprandial glucose levels both in diabetic and regular adults (effect sizes are often larger in pre-diabetic and diabetic individuals, as is often the case here).

Allulose blood glucose and insulin areas under the curve in comparisons with other sugars (source: figure 2)

But wait, there’s more!

Several studies also appear to show lower plasma triglyceride levels and improved lipid profiles (perhaps via the lowering of hepatic lipogenic enzyme activity, maybe involving SCARB1, but probably many others as well), decreased feeding (perhaps via agonizing glucagon-like peptide-1), enhanced fat oxidation, and a reduction in inflammation related to adipokine and cytokine plasma levels (one paper claims this is partially due to down-regulating gm12250 in mice, but if this applies to humans it may be a side-effect of more upstream metabolic changes more so than specific agonism/antagonism, although as is the case with most foods, things get absurdly complicated very quickly with the amount of pathways involved).

Allulose resulting in reduced feeding in high-fat diet obese and diabetic mice (source: figure 3)

It’s worth noting that several of the above studies (particularly ones that attempt to hone in on specific mechanisms of action) are in mice, and in fact, we could go much further if we want to look at mice; it’s trivial to find many more potentially favorable results such as “Not only metformin, but also D-allulose, alleviates metabolic disturbance and cognitive decline in prediabetic rats” or “D-allulose provides cardioprotective effect by attenuating cardiac mitochondrial dysfunction in obesity-induced insulin-resistant rats“. Although there is less (and sometimes conflicting) evidence for e.g. improved lipid profiles in humans, there is certainly more than sufficient evidence of allulose’s effect on reducing blood glucose levels and overall calories consumed, from which we would naturally expect many other beneficial effects to follow. Searching for allulose on pubmed results in a wonderful selection of studies showing very consistent outcomes in this area, and it thus seems plausible that, at the very least, we would see significant reductions in diabetes and obesity if allulose were to be more widely adopted in consumer food products.

Conclusion

In general it seems like replacing sugar with allulose will result in fewer calories consumed, a lower risk of obesity, lower blood glucose (average and area under the curve, sometimes peak) levels and thus improved insulin resistance and a lower risk of diabetes and metabolic syndrome, and potentially some other beneficial effects (which may or may not apply in humans, but if allulose improves your diet and lowers your food intake, I would not be surprised to see improved lipid profiles and a reduction in inflammation, even if entirely for indirect reasons, e.g. cooking at home with allulose instead of purchasing processed foods from the store. It’s also worth noting that while some of these benefits are a direct result of allulose consumption, many are also partially from a reduced intake of sugar and calories – similar to how cutting down on your sugar intake would offer many benefits).

It’s quite possible that if a notable fraction of other sugars in our diet were to be replaced with allulose, the amount we would gain both in QALYs and dollars saved via the resulting reduced healthcare burden would be extremely favorable. Allulose is still relatively new to the market, and as it is also much more expensive than sugar or corn syrups, its future market penetration may be relatively limited by consumer preferences. Regardless of its presence in our broader food ecosystem, you can start experimenting with it yourself today! (Amazon search results page link, in case this saves you 10 seconds)

I usually use allulose to sweeten drinks, greek yogurt, and sometimes add it to sauces or baked goods in small quantities. I’m also pretty interested in glycine and think it may be something that most of us should be having a lot more of as well (some notes on this in the glycine section on my supplements page), but consider it outside the scope of this article for now. Lastly, if the idea of significantly reducing the glycemic index of your meals is appealing, I strongly suggest looking into acarbose – it is a much stronger inhibitor of alpha-glucosidase, well-tolerated, and also relatively cheap.

If you enjoyed this article you might also enjoy my supplements page which discusses many other ingredients and drugs that I find interesting with respect to longevity. Feel free to reach out with any comments or corrections via any communication method on my about page, thanks for reading!

The Bouba/Kiki Effect And Sound Symbolism In CLIP

The bouba/kiki effect is the phenomenon where humans show a preference for certain mappings between shapes and their corresponding labels/sounds.

One of the above objects shall be called a bouba, and the other a kiki

The above image of 2 theoretical objects is shown to a participant who is then asked which one is called a ‘bouba’ and which is called a ‘kiki’. The results generally show a strong preference (often as high as 90%) for the sharply-pointed object on the left to be called a kiki, with the more rounded object on the right to be called a bouba. This effect is relatively universal (in languages that commonly use the phonemes in question), having been noted across many languages, cultures, countries, and age groups (including infants that have not yet learned language very well!), although is diminished in autistic individuals and seemingly absent in those who are congenitally blind.

What makes this effect particularly interesting is less so this specific example, but that it appears to be a general phenomenon referred to as sound symbolism: the idea that phonemes (the sounds that make up words) are sometimes inherently meaningful rather than having been arbitrarily selected. Although we can map the above two shapes to their ‘proper’ labels consistently, we can go much further than just that if desired.

Which is a takete, and which is a maluma? Only you can decide.

We could, of course, re-draw the shapes a bit differently as well as re-name them: the above image is a picture of a ‘maluma’ and a ‘takete’. If you conformed to the expectations in the first image of this section, it’s likely that you feel the maluma is the left shape in this image as well.

We can ask questions about these shapes that go far beyond their names too; which of these shapes is more likely to be calm, relaxing, positive, or explanatory? I would certainly think the bouba and maluma are all four of those, whereas the kiki and takete seem more sharp, quick, negative, or perhaps even violent. If I was told that the above two shapes were both edible, I can easily imagine the left shape tasting like sweet and fluffy bread or candy, while the right may taste much more acidic or spicy and possibly have a denser and rougher texture.

Sound symbolism

The idea that large sections of our languages have subtle mappings of phonemes to meaning has been explored extensively over time, from Plato, Locke, Leibniz, and modern academics, with different figures suggesting their theorized causes and generalizations.

Some of my favorite examples of sound symbolism are those found in Japanese mimetic words: the word jirojiro means to stare intently, kirakira to shine with a sparkle, dokidoki to be heart-poundingly nervous, fuwafuwa to be soft and fluffy, and subesube to be smooth like soft skin. These are some of my favorite words across any language due to how naturally they seem to match their definitions and how fun they are to use (more examples because I can’t resist: gorogoro may be thundering or represent a large object that begins to roll, potapota may be used for dripping water, and kurukuru may be used for something spinning, coiling, or constantly changing. There are over 1,000 words tagged as being mimetic to some extent on JapanDict!).

For fun I asked some of my friends with no prior knowledge of Japanese some questions about the above words, instructing them to pair them to their most-likely definitions, and their guesses were better than one would expect by random chance (although my sample size was certainly lacking for proper scientific rigor). The phonestheme page on Wikipedia tries to give us some English examples as well, such as noting that the English “gl-” occurs in a large number of words relating to light or vision, like “glitter”, “glisten”, “glow”, “gleam”, “glare”, “glint”, “glimmer”, “gloss”. It may also be worth thinking about why many of the rudest and most offensive words in English sound so sharp, often having very hard consonants in them, or why some categories of thinking/filler words (‘hmm’… ‘uhhh…’) sound so similar across different languages. There are some publications on styles of words that are found to be the most aesthetically elegant, including phrases such as ‘cellar door’, noted for sounding beautiful, but not having a beautiful meaning to go along with it.

Sound Symbolism in Machine Learning with CLIP

I would guess that many of the above aspects of sound symbolism are likely to be evident in the behavior some modern ML models as well. The reason for this is that many recent SOTA models often heavily utilize transformers, and when operating on text, use byte-pair encoding (original paper). The use of BPE allows the model to operate on textual input smaller than the size of a single word (CLIP has a BPE vocab size of 41,192), and thus build mappings of inputs and outputs between various subword units. Although these don’t correspond directly to phonemes (and of course, the model is given textual input rather than audio), it’s still likely that many interesting associations can be found here with a little exploration.

To try this out, we can use models such as CLIP+VQGAN or the more recent CLIP-guided diffusion, prompting them to generate an image of a ‘bouba’ or a ‘kiki’. One potential issue with this is that these words could have been directly learned in the training set, so we will also try some variants including making up our own. Below are the first four images of each object that resulted.

four images of “an image of a bouba | trending on artstation | unreal engine”
four images of “an image of a kiki | trending on artstation | unreal engine”

The above eight images were created with the prompt “an image of a bouba | trending on artstation | unreal engine”, and the equivalent prompt for a kiki. This method of prompting has become popular with CLIP-based image generation models, as you can add elements to your prompt such as “unreal engine” or “by Pablo Picasso” (and many, many others!) to steer the image style to a high-quality sample of your liking.

As we anticipated, the bouba-like images that we generated generally look very curved and elliptical, just like the phonemes that make up the word sound. I have to admit that the kiki images appear slightly less, well, kiki, than I had hoped, but nonetheless still look cool and seem to loosely resemble the word. A bit disappointed with this latter result, I decided to try the prompt with ‘the shape of a kikitakekikitakek’ instead, inserting a comically large amount of sharp phonemes all into a single made-up word, and the result couldn’t have been better:

the shape of a kikitakekikitakeki | trending on artstation | unreal engine

Having inserted all of the sharpest-sounding phonemes I could into a single made-up word and getting an image back that looks so amazingly sharp that it could slice me in half was probably the best output I could have hoped for (perhaps I got a lucky seed, but I just used 0 in this case). We can similarly change the prompt to add “The shape of” for our previous words, resulting in the shape of a bouba, maluma, kiki, and takete:

the shape of a bouba (top left), maluma (top right), kiki (bottom left), and takete (bottom right)

It’s cool to see that the phoneme-like associations within recent large models such as CLIP seem to align with our expectations, and it’s an interesting case study that helps us imagine all of the detail that is embedded within our own languages and reality – there’s a lot more to a word than just a single data point. There’s *a lot* of potential for additional exploration in this area and I’m definitely going to be having a lot of fun going through some similar categories of prompts over the next few days, hopefully finding something interesting enough to post again. If you find this topic interesting, some words you may want to search for along with their corresponding Wikipedia pages include: sound symbolism, phonestheme, phonaesthetics, synesthesia, ideathesia, and ideophone, although I’m not currently aware of other work that explores these with respect to machine learning/AI yet.

Thanks for reading! I appreciate feedback via any medium including email/Twitter, for more information and contact see my about page