Leveraged and alternative ETFs: Investing with higher risk tolerance and significantly greater potential upsides

I often end up talking about finance a lot, and in doing so often mention investing strategies and asset classes that many regular retail investors aren’t aware of. Although the world of financial derivatives is vast and unknown to most, I wanted to make a brief post about some simple products which I think should have more publicity, primarily that of leveraged ETFs. This post is a brief introduction to some investing strategies that some retail investors choose to use for higher risk tolerance and significantly greater potential performance. This post is not investment advice, in case I need to actually say that. I should also add that suggesting holding leveraged ETFs for longer periods is a relatively controversial view within the wonderful world of dollarmancy; nonetheless I present my own views here honestly should anyone wish to know them.

stonks
this could be you!

Cash is not your friend

At the lowest level of risk tolerance, many choose to simply keep their savings in cash. This is bad when done for longer periods. It is often pointed out that you will slowly lose money to inflation over time (whether that is the 2% inflation rate that the FOMC targets per year, the ~7% rate of 2021, or perhaps much more..), which although true, is not nearly as large of a loss as the opportunity cost incurred by investing in nothing. Many will provide APY estimates for investing in common market indexes between 6% and 9%, but examining as much as the last few decades (or even just the last decade) will show significantly greater numbers. $SPY has returned over 10% per year since its inception 29 years ago, and around 16.5% per year for the last decade (the last 3 years are even more impressive at 26% each on average!). I do not attempt to claim these are indicative of future results, or that we should be promising anyone these numbers, but it does seem to be unfair to weigh our expected market growth by including past decades that go so far back that we lacked not only much of our modern monetary policy knowledge, but also inventions as basic as the Internet itself.

If casual returns of 10.5% per year were not enough to motivate oneself, I often like converting these APYs to the period of a decade – in which case 10.5% corresponds to a 171% gain (1.105^10), 16.5% APY to a 360% gain, and 26% APY to a >900% gain. We could, of course, make these numbers even more grandiose by telling someone what returns they may expect by holding an investment for 20 or 50 years, but I find a decade to be a relative sweet spot, perhaps because people have an easier time imagining themselves a decade in the future rather than several.

I have heard many reasons for why people choose not to invest in ETFs (or anything similar such as individual stocks), from the reasonable “I am purchasing a house in a few months and now is not the time to take on any risk”, to the questionable “I am waiting for things to cool down a bit and I am a bit worried about some things in the near future”, to the absurd “I do not trust wall street or bankers, sorry” (and indeed, much can be said about how poorly we educate our citizens in the US about basic personal finance, which unfortunately involves much more than just basic investing). I am not going to spend many words attempting to convince someone that holding cash long-term (a year or more) is sub-optimal, because it seems obvious enough to me that it’s considered outside the scope of this post.

What margin is and isn’t

Most young professionals are now fully aware of what index funds are, and often have some simple strategies for investing in them. While it’s not my job to decide the risk tolerance of others, I do think it’s nice to at least be aware of some options that can generate significantly higher long-term returns than these traditional index ETFs. This is not investment advice, and regardless of if it was, I would not want to be responsible for someone else’s choices should things turn south.

The primary product I’d like to mention is that of leveraged ETFs. Many will initially recoil upon hearing the term ‘leverage’ mentioned in the context of personal finance, because they know that it’s scary and can be involved in situations where someone loses their entire principle (that is, 100% of their portfolio). It’s for this reason that I want to start with mentioning the difference between buying stocks on margin and purchasing a product which itself uses margin.

Buying stocks on margin is generally considered to be risky, because you are buying more than you can afford with your own money, effectively taking a loan from your broker in order to afford additional shares. Generally leverage of up to 4x is attainable with popular large-cap stocks on most US brokers, although there’s many exceptions to this. Although buying stocks on margin is not something I would generally suggest for many reasons, it does have a lot of uses, and it can be much less intimidating and dangerous than many may guess. Tools to analyze, manage, and properly limit one’s risk to a comfortable level are readily available, and rates for margin loans can be as low as 1% or under (IBKR is generally the golden standard for the lowest margin rates for regular retail investors, but some other platforms do offer better interfaces, tools, or additional products, and will also be able to negotiate rates with you should you have sufficient capital).

The obvious downside to margin is that you can lose much more of your investment. Theoretically, if you bought a stock with 4X leverage and it then declines by 25%, you would find yourself broke. In practice, you will get liquidated by your broker before this happens, unless the 25% decline happens instantaneously and they do not have enough time to sell your securities on your behalf (If you have heard the term margin call before, that is what happens when you do not have enough capital to maintain your leverage, generally after whatever you own performs very poorly. You can either deposit more money to get back to your maintenance margin, sell some of the products you own via leverage, or let your counterparty liquidate them for you). I am not going to get into the different types of margin or ideal scenarios for using it (of which there are many – remember, this is a loan with an interest rate of only 1%!) in this post, but rather have included this information to help it contrast with what a leveraged ETF is.

Leveraged ETFs

A leveraged ETF is not the same as buying stocks on margin. It is similar in that it is a higher-risk investment that easily allows one to lose or gain much more than usual, but it is different in that you are not taking out a loan explicitly nor implicitly, are not in debt, and therefore cannot be margin called, liquidated, or otherwise lose your shares via any means except via deciding to sell them yourself (this doesn’t mean they can’t still decrease in value by an arbitrary amount of anything less than 100%, however).

A leveraged ETF functions similarly to a regular ETF – it is a security that you can purchase, in which the work of managing your portfolio is abstracted away from you, and instead done by the issuer of the ETF. Instead of buying shares in 500 companies and managing their proportions yourself, you can simply purchase a share of $SPY and forget about it. In exchange for this convenience, you are charged a fee of 0.094% per year (this is often listed by brokers and compiled by ETF websites, but the original source is in the prospectus for the given security). The goal of an ETF is to track its underlying index – if the S&P 500 index is down by 1% in a given day, $SPY should be down close to that amount as well. A leveraged ETF attempts to perform the same function, however it introduces a linear multiplier which multiplies the intended gains and losses. In the US you will generally only find products that offer 2x or 3x leverage due to SEC regulations (3x products are often grandfathered in, as a 2020 update from the SEC suggests a general cap of 200% leverage via derivatives being allowed), although this introduces much more than enough additional risk and volatility for most investors’ appetites (should one want more leverage, they can create additional artificial leverage through the use of options, but that is also outside the scope of this post. Also, gambling is bad, Just Say Neigh!)

Leveraged ETFs are re-balanced daily, and thus intend only to match the performance of their underlying index (multiplied by 2 or 3) for a given day. If the S&P 500 index goes up 1.5% in a day, then a 2X leveraged ETF for it should return close to 3% that day. Due to their targets being daily, some investors often misinterpret this as being equivalent to matching returns on longer periods, although this is not the case. This has been misunderstood enough that the SEC has an alert attempting to inform investors of this, providing some historical examples of leveraged ETFs declining in value during longer periods, during which the underlying index performed positively. This is generally referred to as ‘volatility drag’, and is one of the largest reasons for which many discourage investors from purchasing these products. Much has been written about it, so I will just offer a very short summary: during periods of volatility, leveraged ETFs will perform worse than one would expect at first glance. To give a simple example as to why, imagine that portfolio A returns 5% on day one and then loses 5% on day two. If you started with $100, you will end up with $99.75 ($100 * 1.05 * 0.95). If portfolio B multiplied these daily fluctuations by 3X and returned 15% on day one and -15% on day two, $100 would turn into $97.75 ($100 * 1.15 * 0.85). As you can imagine, if we iterated over these scenarios many times, portfolio B would start to perform terribly in comparison to the portfolio with less leverage.

Volatility drag, aptly-named, is bad during periods of volatility, but it’s particularly bad when there’s not enough underlying momentum in the upward direction to counteract it during longer periods. During a market that is performing even moderately well, generally the greater returns provided by leveraged products don’t just return more than is lost due to volatility drag, but return so much more that being fearful of the concept can be actively harmful (this is likely a controversial opinion in many areas, for what it’s worth – but many people become scared of an investment that could feasibly return 1,000% over a period because of a potential loss of 10% or 50%, even if it’s clearly a very high expected value. In some cases this may be rational due to the diminishing returns of utility provided by additional capital (money may buy a little happiness, but this caps out pretty quickly, and having no money is definitely much worse than having just a little!), but it is well-known that humans are far too risk-averse as a general principle regardless).

To provide some examples, I will mention some leveraged ETFs alongside the returns that they have provided historically. As usual, past performance is not an indication of future results!

$SPUU, a 2X-daily-leveraged ETF that tracks the S&P 500 index, has returned an average of 32% annually for the last 5 years, and 27% annually since inception. $SPXL, a 3X-daily-leveraged ETF that also tracks the S&P 500, has returned an average of 41% annually for the last ten years. Those of you used to performing basic calculations on compounding annual rates will quickly realize how absolutely insane these numbers are – 41% returns compounding for a decade comes out to a return of +3,000%! This is something that is possible, and that many investors have actually attained, providing they didn’t sell during draw-downs (this is not the same as it being guaranteed, or even probable, however).

If past performance is not a promise of future performance, then why is it being mentioned so saliently here? Because although strong performance is not guaranteed, this helps to illustrate the potential of what happens with leveraged ETFs when things go really well, which we can reasonably say has been the case since 2010 to 2022. Because things are not guaranteed to go well, putting 100% of your net worth into these leveraged products is reckless and is very likely a bad idea. However, just as some people like to have hedges just in case things go south, I think it’s important to have some minor positions in place just in case the opposite occurs: If we get lucky and the next 10 years go as well as the last, it is quite possible to attain a 20x, 30x, or greater return on your investment. If you get unlucky, you may lose some or most of your investment, but no more than 100% of it, so the risk to reward is very strongly in your favor (yes, the math is much more complicated than this, but the result holds in more nuanced conditions regardless). In the next section I will go over a few basic common questions about leveraged ETFs, as well as mentioning more of the negatives.

Leveraged ETFs exist for most popular stock indexes, including sector indexes. For example, $SOXL is a 3X-leveraged ETF based upon the ICE Semiconductor Index, which primarily consists of companies related to semiconductor manufacturing. As it is my personal opinion that we are going to tile the world several times over with semiconductors (or something equivalent) in the coming decades, this is a product that I’m a fan of personally, even if it is very high-risk. For some listings of leveraged ETF products, check out out these pages from Direxion and Proshares

Responses to common concerns about leveraged ETFs

Aren’t leveraged ETFs not intended to be held for longer than a day?

This is mentioned in many locations, but it functions primarily for the purposes of legal liability and investor protection. There is nothing wrong with holding these products for longer periods, as long as one is properly educated about them. This is the type of warning where those that it does not apply to will know they can ignore it. There are other similarly-accessible products that are much worse ideas to hold for longer durations, for example inverse-leveraged ETFs, which return the opposite of what the underlying index returns, and thus trend towards zero over the long-run (for an example, $SPXS has returned -47.22% since inception, which leads to over a 99% loss after a decade. If you’re curious why inverse ETFs exists, they are primarily for short-term speculation and various types of hedging).

Aren’t leveraged ETFs subject to volatility drag, and thus a bad idea to hold long-term?

As mentioned above, volatility drag is an important thing to be educated about and aware of. However, if markets actually perform well, the potential gains from leveraged ETFs significantly outweigh (often by more than an order of magnitude) losses due to volatility drag. Regardless, it is worth noting that as many leveraged ETFs are recent financial products, there is an inherent cherry-picking present in the data used to show how well they perform, as the previous 5-20 years have been favorable financially for most US sectors.

Don’t leveraged ETFs have much higher management fees than most normal ETFs?

This is true, and is also something to note. As with the above two examples, $SPUU’s gross expense ratio is 0.88%, and $SPXL’s is 1.03%. Similarly to volatility drag, while it’s important to be aware of these expenses as they do add up and eat into long-term profits, if the market performs well, you will make so much that you will not even notice it.

I don’t want to get margin called, gamble with money that is not mine, or be in debt

Luckily none of these things occur when purchasing leveraged ETFs. You can still lose almost all of your money, but you cannot go into debt or have your shares taken away from you (unless you are engaged in other things that may cause this).

Leveraged ETFs have draw-downs that are far too high for the risk tolerance of every day people

I would say this is completely true. If we take a fund like $SPXL and look at what happened during the covid crash, it crashed from $76 to $18 in a single month, or a decline of around 77%. Apart from this being bad financially, drawdowns this large often cause significant emotional distress to investors and can easily cause them to make poor choices and panic-sell at market bottoms. While $SPXL may have returned back to $76 in less than a year (and then somehow doubled in the year after that..), this will obviously not always be the case. It’s quite possible for drawdowns in some leveraged ETFs to reach 90% or more, even if very rare.

This is gambling

All investing is gambling, mathematically speaking. The absence of investing is also gambling due to opportunity cost – if you hold USD, you are literally betting for it and the US to do well! While it’s true that this is more like gambling than other financial products in the views of many, it should not be compared to acts such as buying a lottery ticket or going to a casino, where there is a known large house edge against you, with the objects in question having been specifically constructed in order to gain the upper hand over you.

Markets exists everywhere and will not go away any time soon, so there is no option of ‘not playing’ the game, as unfortunate as that may be for some of us. The only question is what one’s risk tolerance and personal choices are, not whether they exist or not, because they are forced into existence by our environment. While it may be easy to lose a lot of money on leveraged ETFs, it is nowhere near as bad as buying short-term out-of-the-money options, binary options, 100x leveraged cryptocurrency swaps, 250x forex trades, writing uncovered cryptocurrency options, and many other ‘fun’ products that exist and are often traded by young males addicted to gambling.

How much of my money should I invest into leveraged ETFs?

I have no clue; the right answer for you, dear reader, could very well be 0%, 100% or anywhere in between, but I am not the one that can decide for you. I can say that it is worth your time to learn a lot about how personal finance works however, regardless of your risk tolerance or intentions.

Something something trading leveraged-ETFs or other things

Although I am not in the business of telling people what to do financially, I do enjoy telling people things I think that they should not do, and one of those is ‘trading’. The short version of my advice on this matter is that you should be buying and selling things as infrequently as possible, and you should avoid things like ‘day trading’ like the plague. If you find yourself constantly checking prices, you are likely over-leveraged. I have watched too many bad things happen to too many amazing people, many of them very smart, and most of them young males, and I want to do what I can to cause gambling addictions and casual day-trading to happen less. The humor of places like r/wallstreetbets may be quixotically funny at times and comically sardonic at others, but behind all of the fun people are having with memes about cryptocurrencies and options on Reddit and Twitter, lay thousands of people who have lost their life savings, many of which who end up taking their own lives or losing decades of accumulated capital. Markets are not a game, and if they find a way to eat you alive, they will, as they have become exceedingly efficient at it in the recent few decades.

Further Reading

Any thoughts, corrections, suggestions? Feel free to say hi on Twitter or provide Anonymous Feedback

Pascalian Longevity: Why not?

Scott Alexander of SlateStarCodex / AstralCodexTen recently wrote Pascalian Medicine, in which he looks at various substances purported to improve covid outcomes, but which have relatively low amounts of evidence in their favor, likening administration of all of them to patients to a Pascal’s wager-type argument: if there is a small probability of a potential treatment helping with covid, and if it’s also very unlikely that this treatment is harmful, should we just give it to the patient regardless of if the quality of evidence is low and uncertain, as it would clearly have a positive expected outcome regardless?

The naive answer to this could simply be to attempt to calculate an expected value (note: I use the term expected value often here, but in some cases the terms hazard ratio, relative risk, or odds ratio would be more appropriate) for each treatment, and administer it if it’s positive. But there could be some unintended consequences of using this methodology over the entire set of potential treatments: we could end up suggesting treatments of 10 or 100+ pills for conditions, and apart from something just feeling off about this, it could magnify potential drug interactions, some treatments could oppose others directly, the financial cost could start to become prohibitive, and it could decrease patient confidence and have many other undesirable second-order effects.

Pascalian Longevity

There are many counter-arguments presented to the above concept which become less salient when the goal is changed from ‘find drug treatments to prescribe to all covid patients’ to ‘find personal health interventions that increase your own lifespan/longevity’.

I am fortunate enough that I am able to evaluate potential longevity interventions myself, pay for them myself, administer them myself, and review their potential effects on me myself. I might not do a perfect job of this – research is difficult, time-consuming, and lacking in rigor and quantity, and finding appropriate longevity biomarkers to quantitatively asses the effects of interventions is also difficult. But uncertainty is a given here, and that is why we incorporate it into our frameworks when deciding if something is worth doing or not by calculating an expected value. Furthermore, any harm that I may accidentally incur will only be done to myself, reducing the ethical qualms of this framework to near-zero (I would strongly oppose arguments that I should not have the right to take drugs which I think may significantly improve my own health, although some may disagree here).

My modus operandi with respect to longevity may have many uncertainties in its output, but still operates with a very strong (in my opinion) positive expected value: If a substance significantly and consistently increases the lifespan of organisms similar to humans (ideally in humans), and is also very safe in humans, then it is something that I want to take

This is how I operate personally with longevity, and it does result in me taking quite a few things (currently I’m at around 15). I do still try to minimize what I take as a meta-principle (for example, setting a minimum threshold of expected value that a substance must provide to warrant inclusion, rather than simply accepting any positive expected value) for a few reasons: firstly, to reduce potential drug interactions (which we do attempt to asses on a per-substance basis, rather than account for as an unknown, but unknowns are unfortunately a very large component of messing with biology regardless). Secondly, to keep my costs relatively sane, although I am not too worried about this as there are few ways to spend money more effectively than on trying to improve your health. Thirdly, to reduce the occurrence of interventions that may have the same or opposing mechanisms of action (taking two things with the same mechanism of action may be okay, but sometimes dose-response curves are less favorable, and taking >~2x of something will result in diminished or even negative returns). Lastly, to minimize potential secondary side-effects that could be cumulative over large classes of substances (for example, effects on the liver).

I don’t intend to promote any specific substances or interventions here as I don’t give medical advice, nor do I want anything specific to be the focus of this post, but I do want to remind us that just as we can calculate expected values in a utilitarian fashion and get effective altruism as a result, we can do the same for longevity interventions and get a very strong chance at notably increasing our lifespan/healthspan as a result. I do have a list of some of what I take here, but it is definitely not intended to promote anything specific to others.

Why Not?: Potential counter-arguments

Algernon’s Law

Algernon’s Law is sometimes brought up, suggesting that evolution has already put a lot of effort into optimizing our body, and thus we are unlikely to find improvements easily. But, as Gwern notes in the above link, there’s at least three potential ways around this reasoning: interventions may be complex (and/or too far away in the evolutionary plane) and could not have easily been found, they may be minor or only work in some individuals, or they may have a large trade-off involved and cause harm to reproductive fitness.

Although some areas of future longevity treatments may fall under exception one and be complex enough that evolution could not have found them, I would suggest that the majority of today’s potential treatments fall under exception three: evolution optimizes for reproductive fitness, not for longevity, and for this reason there are many interventions which will improve our longevity that it has not given to us already (this is part of why I am more optimistic about longevity interventions than I am about intelligence interventions/nootropics).

For an extreme example of this, it has been noted that castrated males often live longer, and that this is obviously something evolution would not be very interested in exploring. Although this has been found with median lifespan in male mice (maybe in females too?), there is also purported historical data on Korean eunuchs suggesting that they may have lived a full 14-19 years longer (there are definitely potential confounding variables and/or bad data here, but we don’t have RCTs on this in humans for obvious reasons..), and a more recent study in sheep that is also highly relevant: Castration delays epigenetic aging and feminizes DNA methylation at androgen-regulated loci, where epigenetic aging clocks that look at DNA methylation are used in castrated sheep. There are other traits that seem to improve longevity as well, for example decreased height. It seems quite plausible that there are a lot of trade-offs that optimize for strong reproductive fitness early in the lifespan of organisms, which end up costing the organism dearly in terms of longevity. These trade-offs may be involved in many areas such as testosterone, estrogen, growth hormone, IGF-1, caloric restriction, mtor activation, and many others.

Large error in estimating unknown risks

One other counter-argument here is often along the lines of “you are messing with things you don’t understand, and you could be hurting yourself but be unaware of this; the damage may also be difficult to notice, or perhaps only become noticeable at a much later time”

It is true that our understanding of biology is lacking, and therefore also that we are operating in highly uncertain environments. I would be open to evidence that suggests reasoning for why we may be systemically underestimating the unknown risks of longevity interventions, but given how strong the potential upside is, these would have to be some pretty terrible mistakes that are being made. It is often noted how curing cancer may only extend human lifespan by a few years, whereas a longevity improvement of 5% for everyone would provide much more value (and is also much easier to find in my opinion). One could make an argument here that even if I was doing something that notably increased my risk of e.g. cancer, if the expected lifespan increase of this intervention was as much as 1-5%, this could still be a huge net positive for my health! I don’t take approaches that are this extreme regardless, and I try to keep the risk side of my risk/reward ratio low independently of the level of potential reward in attempt to account for this uncertainty. I am also not aware of many interventions that seem to have very high numbers in both the numerator and denominator here, although I am pretty certain that they do exist; I don’t currently take anything that I think has notably detrimental side-effects for the time being.

Is it fair to call this approach Pascallian?

The original nature of Pascal’s wager is that of extreme probabilities resulting in positive expected values, but the numbers that we are operating with are nowhere near as extreme as they could be. It is probably not a good idea to take 10,000 supplements, each of which have a 0.1% chance of extending your lifespan by a year for many reasons (similarly, if 10,000 people that claimed to be God all offered me immortality for a small fee, I would hope to decline all of their offers unless sufficient evidence was provided by one).

As I’m not arguing in favor of taking hundreds or thousands of supplements in the hopes that I strike gold with a few of them, it may be worth noting that ‘Pascallian Longevity’ would be a poor label for my strategy. Regardless, taking just 5-10 longevity interventions with a strong upside potential seems to be significantly more than almost everyone is doing already, so I still stand by my claim that there are many free lunches (free banquets, if you ask me) in this area, and I am very optimistic about the types of longevity interventions we’ll find in the coming decades.

Open to any corrections/comments on Twitter or any medium on my about page

My Favorite Links

This page contains a collection of some of my favorite links (mostly blog posts), roughly categorized by author. My hope is that others similar to myself can waste enjoy countless hours of reading from recursively following some of the links here. I haven’t yet finished this post but have published it regardless.

Scott Alexander (Twitter): As the author behind SlateStarCodex (now AstralCodexTen) and many great LessWrong posts, in my opinion Scott is among one of the best written content creators of the last decade. He focuses primarily on psychiatry, rationality, meta-science, critical thinking, and logical reasoning. Favorite posts:

  • todo (this will take me around 20-50 hours if I work well, hence not having finished it yet, maybe some day!)

Gwern Branwen (Twitter – currently private): Well-known for having quality deep dives in diverse areas such as statistics, technology, machine learning, genetics, psychology, and many others. Also often recognized as an amazingly aesthetic, verbose, and highly-usable website. Favorite posts:

Scott Aaronson: A theoretical computer scientist with a focus on quantum computing and complexity theory. Although his posts on quantum computational complexity theory research go over my head, I’ve enjoyed some great content from him in other categories. Favorites:

Matt Levine (Twitter): An ex-Goldman Bloomberg opinion columnist with some wonderfully insightful and hilarious posts (offered as a free newsletter, generally ~4x a week) on the happenings in our modern yet often-insane financial world. Posts are generally centered around current events and are best read as they come out. Some examples:

Nintil (Twitter): A wonderful blog by Jose Luis Ricón with a focus on longevity, economics, and meta-science. Favorite posts:

Patrick Collison (Twitter): The CEO and co-founder of Stripe, often with focuses involving meta-science, individual and societal productivity, and economics

  • Fast: Examples of people quickly accomplishing ambitious things together
  • Questions: A short list of interesting questions
  • Advice: Advice, particularly for young and ambitious individuals
  • Book Recommendations: A well-sized list of suggested reading

Sam Altman (Twitter): The CEO of OpenAI and former president of Y Combinator, his posts often focus on startups, artificial intelligence, productivity, and science. Favorites:

  • How to be Successful: Thirteen thoughts on how to achieve long-term successful outcomes: learn a lot, compound yourself, work hard, and be ambitious
  • Productivity: Various productivity tips, such as ‘Picking the right thing to work on is the most important element of productivity and usually almost ignored. So think about it more!’
  • Advice for Ambitious 19 Year Olds: Advice for young and ambitious individuals, such as ‘The best people always seem to be building stuff and hanging around smart people’
  • How to Invest In Startups: Advice about being a good startup investor
  • Super successful companies: Notes some salient commonalities between many very successful companies
  • The Strength of Being Misunderstood: You should trade being short-term low-status for being long-term high-status

Paul Graham (Twitter): The founder of Y Combinator, with many posts focusing on startups, ideas and frameworks for everyday life, as well as advice and reflections for people that fit the founder/builder/nerd stereotype. Some favorites:

  • Do Things That Don’t Scale: An amazing tip on gaining initial traction and leverage by doing high-impact activities that won’t scale, but that will work effectively for the time being
  • What You Can’t Say: Reflections on that which exists outside of the Overton window
  • How to Make Wealth: An essay on effectively building wealth over time
  • Keep Your Identity Small: On why politics and religion yield such uniquely useless discussions due to excessive involvement with personal identity
  • Having Kids: Personal experiences and thoughts on having kids
  • It’s Charisma, Stupid: A 2004 essay arguing that charisma is the most important trait for elected politicians, using the US presidency as an example
  • What I worked an: A personal and emotional memoir on pg’s professional and personal history

Alexey Guzey (Twitter): Currently working on New Science, Alexey has some great blog posts with a focus on properly using the Internet for social leverage (reach out to people more, cold email people more, initiate conversations more, and create content more!), meta-science, productivity, biology, and more. Some favorites:

Melting Asphalt (Twitter): Written by Kevin Simler (along with Robin Hanson (Twitter), co-author of The Elephant in the Brain), Melting Asphalt has a wonderful collection of posts on evolutionary psychology, game theory, and novel and introspective takes on what makes us human. Favorites:

  • Neurons Gone Wild: A beautifully speculative post that suggests a recursively selfish model of biological neurons which enables selfish sub-agents and networks to co-exist in an evolutionary semi-competitive environment within our own minds. Probably my favorite post on this blog for several reasons. Also see Hallucinated Gods
  • Music in Human Evolution: A great book review of Why Do People Sing?: Music in Human Evolution by Joseph Jordania, involving predatory defense mechanisms, disposition of the dead, battle trances, and the audio-visual intimidation display
  • Crony Beliefs: On beliefs that stick around when they shouldn’t
  • Personality: The Body in Society:
    What is personality? ‘Nature and nurture work together to create a prototype, which then negotiates with the external world. The result is a strategy for getting along and getting ahead — a strategy we call “personality”, in other words, ‘Personality is a strategy for making the most of one’s particular lot in life.’ See also: part two and part three
  • Ads Don’t Work That Way: On ‘cultural imprinting’ and signaling in advertising
  • Doesn’t Matter, Warm Fuzzies: Discusses many interesting aspects of human ecology and society, with a focus on rituals, culture, confabulation, mimicry, and more
  • Social Status: Down the Rabbit Hole: On social status in humans, including an analysis of two proposed separate status systems: dominance/submission and prestige/admiration. See also: Social Status II: Cults and Loyalty
  • Border Stories: Borders are a necessary precondition for agency within a hostile ecosystem

Qualia Computing: With a subtitle of ‘revealing the computational properties of consciousness’, Qualia Computing is a great blog for anyone interested in the neurology, phenomenology, and interesting attempts at quantifications and explanations behind our own conscious experiences (qualia)

Patrick Mckenzie (Twitter): An entrepreneur and writer that lives in Japan and currently works at Stripe with a focus on startups and outreach, Patrick has many invaluable posts about finance, startups, marketing and professional communication, and highly-regarded SaaS and entrepreneurial advice. Favorite posts:

Fantastic Anachronism (Twitter): todo, see Recommended Reading

Applied Divinity Studies: todo

Peter Attia (Twitter): todo

Vitalik Buterin (Twitter): todo, see The bulldozer vs vetocracy political axis

Nick Cammarata (Twitter): todo

Lesswrong: todo

Austin Allred: How to skip the resume pile and get a ton of interviews (even without a degree): The ridiculously detailed guide

Overcoming Bias: todo, ‘This is a blog on why we believe and do what we do, why we pretend otherwise, how we might do better, and what our descendants might do, if they don’t all die’, from Robin Hanson.

Aella (Twitter): Popular Posts, Becoming A Whorelord: The Overly Analytical Guide To Escorting, Handling Accusations In Communities, The Polyamory Post

Tim Ferris: 11 Reasons Not to Become Famous

Dynomight (Twitter): todo, see Better air is the easiest way not to die by The impact of air pollution on health is often significantly underrepresented, and working on improving the quality of air in your dwelling can result in a very high ROI for your health

Marginal Revolution: todo

Zvi: todo

(about 15 other todos currently excluded)

Allulose: The Best Sugar Substitute

Allulose (sometimes D-psicose) is by-far one of the best ways to add sweetness to home-cooked meals in a healthy and low-calorie way. As an epimer of fructose, it has been steadily gaining popularity within the last few years, and not without good reason! Allulose is not only nearly calorie-free, but also decreases blood glucose levels with meals, and seems to have a wide range of potentially beneficial effects.This post is a short summary of why allulose is so appealing over sugar and other sugar substitutes.

70% as sweet; 100% as white and crystalline

Overview of Allulose

Allulose is found naturally in wheat, figs, raisins, maple syrup, and molasses, although in relatively trace amounts. It has around 10% the calories of traditional sucrose and can be manufactured from fructose. It’s around 70% as sweet as sucrose (regular sugar), but has a similar taste and feel, which is a large factor behind why it makes a great substitute (or partial substitute) for baking or dissolving into things. The taste of Allulose has a more natural and relaxing quality than some other sugar-replacement options such as xylitol and erythritol, which are both sugar alcohols, but generally have a ‘cooling effect’ (often likened to the aftertaste of consuming mint, which allulose conveniently lacks).

Allulose is also an actual sugar (not a sugar alcohol or other compound), and has similar browning properties to sucrose via the Maillard reaction. One downside to mention is that it does seem challenging to keep some styles of baked goods crunchy with allulose as the only sugar; while it definitely seems to be one of the best options for sweetening drinks, yogurts, ice creams, cakes, and so on, it may not be the best option for super-crunchy cookies (although can make great softer ones!). This seems to be due to allulose not crystallizing when it cools, its ability to hold more moisture, and that it is more soluble in liquids than sucrose; hence it being a great fit for drinks, sauces, and spongy baked goods.

Allulose was designated as GRAS by the FDA in 2019, so is still relatively new to the market compared to many other sugar substitutes, although has been gaining significant popularity for the short period that it has been available for general usage in foods. I’m sometimes now able to find allulose for sale in a supermarket or included in a sweet good (and it is also now being used in products such as Soylent), although its usage is still a small fraction to that of sugar and corn syrups. It can easily be purchased on Amazon for around $10 per lb (regular sugar is generally closer to $1-2 per lb, so it is quite a bit more expensive if you happen to use very large amounts of sugar).

What Sets Allulose Apart

Why might we want alternative sources of sweetness from sucrose to begin with? Although much has been said about the ways sugars are (in some cases) potentially harmful, it seems reasonable to posit that there are two qualities of a diet with high sugar content (remember, this means any typical western diet!) that are undesirable: firstly, the high caloric content of sugar, which makes over-eating significantly easier and therefore contributes to obesity, and secondly, the effects of sucrose on blood glucose levels and thus insulin resistance, which contributes to diabetes and metabolic syndrome.

As we would hope from an alternative to sucrose, allulose doesn’t cause an increase in blood sugar. The reason for this is that it is not absorbed and digested by the gastrointestinal tract, but rather processed by intestinal bacteria. For the most part this is a good thing, and is what enables allulose to both be low-calorie and to not be converted to glucose in the blood stream. The downside of this is that for some people, especially if consumed in large enough quantities, it can cause mildly discomforting side effects such as flatulence, subpar digestion, and abdominal discomfort. This is much more likely to occur if you, for example, eat an entire batch of allulose cookies by yourself (who would do such a thing..!?), rather than simply use it to sweeten a drink or a snack. While I haven’t experienced anything negative myself, everyone is certainly very different when it comes to food.

But, it gets much better than this! Allulose not only doesn’t increase your blood sugar, but actually decreases it! It does this by inhibiting alpha-glucosidase (along with several other similar enzymes), which is an enzyme that breaks down starches and disaccharides into glucose (i.e. causes carbohydrates to lead to blood glucose spikes). Other well-known inhibitors of alpha-glucosidase include acarbose, a popular and simple diabetic drug which significantly extends lifespan in mice and has the exact same potential side effect profile as large allulose doses (and in my opinion is probably very good for most people to be taking, perhaps extending human lifespan via the same mechanism of action as in mice), and sweet potatoes (source, another source). Thus, adding allulose to meals that contain carbohydrates will result in less of a blood glucose spike than if allulose had been excluded.

Comparison of blood glucose area under curve for small quantities of fructose vs allulose (source: figure 1)

There’s now quite a few studies showing this in humans (and dogs and mice!), with allulose consistently attenuating the postprandial glucose levels both in diabetic and regular adults (effect sizes are often larger in pre-diabetic and diabetic individuals, as is often the case here).

Allulose blood glucose and insulin areas under the curve in comparisons with other sugars (source: figure 2)

But wait, there’s more!

Several studies also appear to show lower plasma triglyceride levels and improved lipid profiles (perhaps via the lowering of hepatic lipogenic enzyme activity, maybe involving SCARB1, but probably many others as well), decreased feeding (perhaps via agonizing glucagon-like peptide-1), enhanced fat oxidation, and a reduction in inflammation related to adipokine and cytokine plasma levels (one paper claims this is partially due to down-regulating gm12250 in mice, but if this applies to humans it may be a side-effect of more upstream metabolic changes more so than specific agonism/antagonism, although as is the case with most foods, things get absurdly complicated very quickly with the amount of pathways involved).

Allulose resulting in reduced feeding in high-fat diet obese and diabetic mice (source: figure 3)

It’s worth noting that several of the above studies (particularly ones that attempt to hone in on specific mechanisms of action) are in mice, and in fact, we could go much further if we want to look at mice; it’s trivial to find many more potentially favorable results such as “Not only metformin, but also D-allulose, alleviates metabolic disturbance and cognitive decline in prediabetic rats” or “D-allulose provides cardioprotective effect by attenuating cardiac mitochondrial dysfunction in obesity-induced insulin-resistant rats“. Although there is less (and sometimes conflicting) evidence for e.g. improved lipid profiles in humans, there is certainly more than sufficient evidence of allulose’s effect on reducing blood glucose levels and overall calories consumed, from which we would naturally expect many other beneficial effects to follow. Searching for allulose on pubmed results in a wonderful selection of studies showing very consistent outcomes in this area, and it thus seems plausible that, at the very least, we would see significant reductions in diabetes and obesity if allulose were to be more widely adopted in consumer food products.

Conclusion

In general it seems like replacing sugar with allulose will result in fewer calories consumed, a lower risk of obesity, lower blood glucose (average and area under the curve, sometimes peak) levels and thus improved insulin resistance and a lower risk of diabetes and metabolic syndrome, and potentially some other beneficial effects (which may or may not apply in humans, but if allulose improves your diet and lowers your food intake, I would not be surprised to see improved lipid profiles and a reduction in inflammation, even if entirely for indirect reasons, e.g. cooking at home with allulose instead of purchasing processed foods from the store. It’s also worth noting that while some of these benefits are a direct result of allulose consumption, many are also partially from a reduced intake of sugar and calories – similar to how cutting down on your sugar intake would offer many benefits).

It’s quite possible that if a notable fraction of other sugars in our diet were to be replaced with allulose, the amount we would gain both in QALYs and dollars saved via the resulting reduced healthcare burden would be extremely favorable. Allulose is still relatively new to the market, and as it is also much more expensive than sugar or corn syrups, its future market penetration may be relatively limited by consumer preferences. Regardless of its presence in our broader food ecosystem, you can start experimenting with it yourself today! (Amazon search results page link, in case this saves you 10 seconds)

I usually use allulose to sweeten drinks, greek yogurt, and sometimes add it to sauces or baked goods in small quantities. I’m also pretty interested in glycine and think it may be something that most of us should be having a lot more of as well (some notes on this in the glycine section on my supplements page), but consider it outside the scope of this article for now. Lastly, if the idea of significantly reducing the glycemic index of your meals is appealing, I strongly suggest looking into acarbose – it is a much stronger inhibitor of alpha-glucosidase, well-tolerated, and also relatively cheap.

If you enjoyed this article you might also enjoy my supplements page which discusses many other ingredients and drugs that I find interesting with respect to longevity. Feel free to reach out with any comments or corrections via any communication method on my about page, thanks for reading!

The Bouba/Kiki Effect And Sound Symbolism In CLIP

The bouba/kiki effect is the phenomenon where humans show a preference for certain mappings between shapes and their corresponding labels/sounds.

One of the above objects shall be called a bouba, and the other a kiki

The above image of 2 theoretical objects is shown to a participant who is then asked which one is called a ‘bouba’ and which is called a ‘kiki’. The results generally show a strong preference (often as high as 90%) for the sharply-pointed object on the left to be called a kiki, with the more rounded object on the right to be called a bouba. This effect is relatively universal (in languages that commonly use the phonemes in question), having been noted across many languages, cultures, countries, and age groups (including infants that have not yet learned language very well!), although is diminished in autistic individuals and seemingly absent in those who are congenitally blind.

What makes this effect particularly interesting is less so this specific example, but that it appears to be a general phenomenon referred to as sound symbolism: the idea that phonemes (the sounds that make up words) are sometimes inherently meaningful rather than having been arbitrarily selected. Although we can map the above two shapes to their ‘proper’ labels consistently, we can go much further than just that if desired.

Which is a takete, and which is a maluma? Only you can decide.

We could, of course, re-draw the shapes a bit differently as well as re-name them: the above image is a picture of a ‘maluma’ and a ‘takete’. If you conformed to the expectations in the first image of this section, it’s likely that you feel the maluma is the left shape in this image as well.

We can ask questions about these shapes that go far beyond their names too; which of these shapes is more likely to be calm, relaxing, positive, or explanatory? I would certainly think the bouba and maluma are all four of those, whereas the kiki and takete seem more sharp, quick, negative, or perhaps even violent. If I was told that the above two shapes were both edible, I can easily imagine the left shape tasting like sweet and fluffy bread or candy, while the right may taste much more acidic or spicy and possibly have a denser and rougher texture.

Sound symbolism

The idea that large sections of our languages have subtle mappings of phonemes to meaning has been explored extensively over time, from Plato, Locke, Leibniz, and modern academics, with different figures suggesting their theorized causes and generalizations.

Some of my favorite examples of sound symbolism are those found in Japanese mimetic words: the word jirojiro means to stare intently, kirakira to shine with a sparkle, dokidoki to be heart-poundingly nervous, fuwafuwa to be soft and fluffy, and subesube to be smooth like soft skin. These are some of my favorite words across any language due to how naturally they seem to match their definitions and how fun they are to use (more examples because I can’t resist: gorogoro may be thundering or represent a large object that begins to roll, potapota may be used for dripping water, and kurukuru may be used for something spinning, coiling, or constantly changing. There are over 1,000 words tagged as being mimetic to some extent on JapanDict!).

For fun I asked some of my friends with no prior knowledge of Japanese some questions about the above words, instructing them to pair them to their most-likely definitions, and their guesses were better than one would expect by random chance (although my sample size was certainly lacking for proper scientific rigor). The phonestheme page on Wikipedia tries to give us some English examples as well, such as noting that the English “gl-” occurs in a large number of words relating to light or vision, like “glitter”, “glisten”, “glow”, “gleam”, “glare”, “glint”, “glimmer”, “gloss”. It may also be worth thinking about why many of the rudest and most offensive words in English sound so sharp, often having very hard consonants in them, or why some categories of thinking/filler words (‘hmm’… ‘uhhh…’) sound so similar across different languages. There are some publications on styles of words that are found to be the most aesthetically elegant, including phrases such as ‘cellar door’, noted for sounding beautiful, but not having a beautiful meaning to go along with it.

Sound Symbolism in Machine Learning with CLIP

I would guess that many of the above aspects of sound symbolism are likely to be evident in the behavior some modern ML models as well. The reason for this is that many recent SOTA models often heavily utilize transformers, and when operating on text, use byte-pair encoding (original paper). The use of BPE allows the model to operate on textual input smaller than the size of a single word (CLIP has a BPE vocab size of 41,192), and thus build mappings of inputs and outputs between various subword units. Although these don’t correspond directly to phonemes (and of course, the model is given textual input rather than audio), it’s still likely that many interesting associations can be found here with a little exploration.

To try this out, we can use models such as CLIP+VQGAN or the more recent CLIP-guided diffusion, prompting them to generate an image of a ‘bouba’ or a ‘kiki’. One potential issue with this is that these words could have been directly learned in the training set, so we will also try some variants including making up our own. Below are the first four images of each object that resulted.

four images of “an image of a bouba | trending on artstation | unreal engine”
four images of “an image of a kiki | trending on artstation | unreal engine”

The above eight images were created with the prompt “an image of a bouba | trending on artstation | unreal engine”, and the equivalent prompt for a kiki. This method of prompting has become popular with CLIP-based image generation models, as you can add elements to your prompt such as “unreal engine” or “by Pablo Picasso” (and many, many others!) to steer the image style to a high-quality sample of your liking.

As we anticipated, the bouba-like images that we generated generally look very curved and elliptical, just like the phonemes that make up the word sound. I have to admit that the kiki images appear slightly less, well, kiki, than I had hoped, but nonetheless still look cool and seem to loosely resemble the word. A bit disappointed with this latter result, I decided to try the prompt with ‘the shape of a kikitakekikitakek’ instead, inserting a comically large amount of sharp phonemes all into a single made-up word, and the result couldn’t have been better:

the shape of a kikitakekikitakeki | trending on artstation | unreal engine

Having inserted all of the sharpest-sounding phonemes I could into a single made-up word and getting an image back that looks so amazingly sharp that it could slice me in half was probably the best output I could have hoped for (perhaps I got a lucky seed, but I just used 0 in this case). We can similarly change the prompt to add “The shape of” for our previous words, resulting in the shape of a bouba, maluma, kiki, and takete:

the shape of a bouba (top left), maluma (top right), kiki (bottom left), and takete (bottom right)

It’s cool to see that the phoneme-like associations within recent large models such as CLIP seem to align with our expectations, and it’s an interesting case study that helps us imagine all of the detail that is embedded within our own languages and reality – there’s a lot more to a word than just a single data point. There’s *a lot* of potential for additional exploration in this area and I’m definitely going to be having a lot of fun going through some similar categories of prompts over the next few days, hopefully finding something interesting enough to post again. If you find this topic interesting, some words you may want to search for along with their corresponding Wikipedia pages include: sound symbolism, phonestheme, phonaesthetics, synesthesia, ideathesia, and ideophone, although I’m not currently aware of other work that explores these with respect to machine learning/AI yet.

Thanks for reading! I appreciate feedback via any medium including email/Twitter, for more information and contact see my about page

Does 17α-estradiol/estrogen extend male human lifespan?

17α-estradiol is a relatively (or completely) non-feminizing form of estradiol (E2), or estrogen. It is a naturally occurring enantiomer of 17β-estradiol (the much more common form of estradiol, usually just referred to as ‘estradiol’) which is found in both male and female humans. This post a a brief essay that discusses the prospect of it extending lifespan in humans. There are two primary types of estrogen receptors, ERα and Erβ, and as you may expect, 17α-estradiol appears to show a stronger binding affinity for ERα. It has a very low binding affinity in locations that generally induce feminization (which appear to be sometimes be both ERα and ERβ), so it’s also possible to take as a male without significantly altering one’s appearance towards the opposite gender. Although we can definitively point to a plethora of effects of regular estrogen, it is difficult to tell what the true purpose of 17α-estradiol is in humans, with Stout et al. (2016) stating “the physiological functions of endogenous 17α-E2 are unclear”. There is evidence it has neuroprotective properties, can help treat Parkinson’s disease, cerebrovascular disease, and much more. This likely involves ER-X, which in turn activates MAPK/ERK and many, many other things down the line (as usual..), but it’s difficult to know for certain. Although these reasons were among the reasons that researchers took into account when deciding to dedicate funding to testing 17α-estradiol in mice for longevity effects, subsequent papers have found more exciting mechanisms of action which are elaborated upon below. For some interesting further reading on this topic that goes into more detail exploring possible mechanisms of action here I’d also suggest reading the following papers: Castration delays epigenetic aging and feminizes DNA methylation at androgen-regulated loci, Hypermethylation of estrogen receptor-alpha gene in atheromatosis patients and its correlation with homocysteine.

17α-estradiol has been found to consistently and significantly extend the median lifespan of male mice, including by the NIH’s Intervention Testing Program, the closest thing we have to a gold standard of longevity RCT experimentation in mice, where three studies are rigorously performed at three separate locations, allowing the results to be instantly compared and reproduced by the two other parties and locations upon completion. Strong et al. (2016) find that 17α-estradiol extends median lifespan of male mice by an average of 19% (26%, 23%, and 9% from the three independent testing sites), and increased the maximum age by an average of 12% (21%, 8%, and 8% from the three testing sites, using the 90th percentile). Harrison et al. (2014) similarly find that median male lifespan was increased by 12%, but did not find an increase in maximum lifespan, and these results have been replicated even more in recent years.

These are some impressive results for such a common and simple endogenous substance! One of the first things we notice is that this effect only applies to males, with female lifespan (both median and maximum) being unaffected. As the substance in question is an estrogen, we can assume that this is either due to female mice already having this benefit, as they already have a sufficient level of it, or that something more complex is at play, and there is a different downstream pathway that is only being activated in males for some reason (more on this later). I had initially assumed the former hypothesis was at least a partial explanation, having known that females consistently live longer than males when it comes to humans, and that this was obviously biological in nature. However, it’s much more complicated in mice as females do not always outlive males, and in fact many times the opposite is true. One meta-analysis (good overview, original book source) finds 65 studies where males lived longer and 51 where females lived longer, with this often depending on the strain of mice used, which varies greatly depending on the type of reseasrch and time period. Regardless, it’s clear there is much more at play in this scenario, and perhaps something special about 17α-estradiol in particular.

Although the ITP studies initially included 17α-estradiol due to the reasons mentioned in the first paragraph, later research such as Stout et al. (2016) has now found that 17α-estradiol not only increased AMPK levels (as some other notable longevity substances such as Metformin also do), but also reduced mTOR activity (complex 1!) in visceral adipose tissue, which is rather reminiscent of Rapamycin, which has extended the lifespan of every organism we have performed an RCT with thus far (and likely can in humans too, if you ask me). In a way, this is significantly more exciting, because it gives us a much more plausible way to explain the lifespan extension effects we are noticing. However, it is also partially a disappointment: if these effects are the real reasons why 17α-estradiol extends male mice lifespan, then this substance may offer us nothing that we do not already have via rapamycin and metformin, among others. The paper also noted that fasting glucose, insulin, and glycosylated hemoglobin were reduced along with inflammatory markers improving. These are similar to the types of positive side effects we would expect from a longevity agent, and the study also notes that no feminization nor cardiac dysfunction occurred.

How do these effects (such as AMPK and mTOR modulation) occur? I don’t know, and apparently neither does anyone else. As is often the unfortunate case in biology, the paper has this to say: “The signaling mechanism(s) by which 17α-E2 elicits downstream effects remains elusive despite having been investigated for several decades”. Perhaps just a few more decades to go and this section will be updated with more information, then. Mann et al (2020) find that male mice without ERα do not benefit from 17α-estradiol, which helps us narrow down the first step by excluding Erβ, ER-X and other less-predictable initial mechanisms. Interestingly, they also note that “both 17α-estradiol and 17β-estradiol elicit similar genomic binding and transcriptional activation of ERα”, which would leave us with the question of why we are focusing on 17α-estradiol specifically, if 17β-estradiol (which is much more common) suffices as well. Importantly, they also seem to think changes in the liver might be involved. Garratt et al. (2018) add that distinct sex-specific changes in the metabolomic profile of the liver and plasma were found, and also notes that the longevity benefit for males disappears post-castration. They first supplement males and females, showing many differences related to metabolism including with amino acids. Then they use castrated males and notice that their profiles are the same as the control group, and thus conclude that they are no longer being positively affected by 17α-estradiol. I am unsure if we should be focusing on the AMKP/mTOR effects (which are very relevant to longevity) or on the liver/metabolic effects (which are also very relevant), or if these are in fact just two different temporal points on the same biological pathway which we don’t yet fully understand, but this helps us connect at least a few more dots.

All of the above sounds exciting, but it’s also all in mice. Sometimes this is useful, as mice are actually quite similar to humans (more so than many may expect), but a lot of it is also less useful or outright misleading. I cannot find a way to take only 17α-estradiol in a safe way as a human, however there is a topical cream of it (alfatradiol) which is used to treat pattern hair loss.

Luckily, one thing that the ITP study found was that 17α-estradiol was among one of the substances that seems to perform well with respect to longevity (if not fully) when given later in life (this has replicated afterwards as well), contrary to some others which have the best effect when started in youth and continued until death. In theory I wouldn’t mind waiting a decade or two until we have a better idea of what is going on here, after which point I would hope we have more fruitful and actionable results (especially in humans); although at the same time there’s likely many reasonable and safe ways we can go about achieving this (hopeful) effect in human males (assigned at birth) already, either via a type of estrogen or an estrogenic drug such as a SERM.

It is worth reminding ourselves that 17α-estradiol is already present in humans, and in both sexes, with women generally having significantly higher levels, as one expects of estrogen. Similarly, regular estrogen binds to both estrogen receptors, including our target, which we now know to be the alpha receptor. Given this, is it possible that just taking regular estradiol (for example, estradiol valerate, which for most purposes ends up biologically equivalent to endogenous estradiol and thus also binds to both primary estrogen receptors) to increase the levels of estrogen is a potential longevity intervention?

This is a difficult question to answer with the data currently available, although there are millions of persons assigned male at birth that are already on various forms of estradiol for various reasons, one of them being to assist in gender transition from male to female. As the lifespan benefit only applied to male (assigned at birth) mice, there would be benefits to analyzing these cohorts for more information, especially if we were able to have DNA methylation clocks used on these groups alongside a control (although this would not be a true RCT, as which persons decide to undergo feminizing HRT would not be random, I suspect we could still get the information we’d want with a good sample size).

There are other potential avenues of statistical analysis that could be attempted here, although they prove to be difficult for various reasons. Most male to female transgender individuals decide to transition earlier in their life, and this was also a particularly uncommon choice to make many decades ago in comparison to the present, so we have very few deaths due to age-related causes that we would be able to analyze to attain a proper hazard ratio. Even if we waited a long time for this (or had this data already), it would be terribly confounded due to the lack of randomization and many potential selection effects. Even so, one of the following must be true:

  • 17α-estradiol does not extend male (assigned at birth) human lifespan
  • 17α-estradiol does extend male (assigned at birth) human lifespan, however this does not apply to most/any transgender (m->f) individuals. This could be due to insufficient dosage, insufficient affinity for the alpha receptor, the inclusion of 17β-estradiol, the common addition of other substances such as anti-androgens, or another unknown factors/confounders
  • 17α-estradiol does extend male (assigned at birth) human lifespan, and this effect therefore does apply to most transgender (m->f) individuals, however we have either failed to notice it completely, or other effects/confounding variables ablate this, for example an increased risk of blood clots from estrogen supplementation (which depends greatly on the route of administration as well as type of estrogen used) or various potential side-effects from anti-androgen usage

Option one is certainly a possibility, as it always is in longevity when all of our studies are only in mice. We could differ too much from mice for the mechanism of action to apply to us (perhaps if it is related to metabolism or some newer subset of liver functionality), or if the mechanism of action is indeed the AMPK/mTOR pathways, perhaps 17α-estradiol does not modulate these in humans as it does in mice. This could have implications for other potential longevity agents such as metformin and rapamycin in humans as well, which also heavily involve these pathways, which could cause these agents to interplay synergistically or perhaps cancel one another out, as there may be no further benefit that can be gained after one of these agents is already taken at the optimal dosage. It is worth noting that many aspects related to AMPK/mTOR and DNA methylation are heavily evolutionary conserved as well (mTOR quite strongly, which is another reason why rapamycin likely extends human lifespan). We also already know that human females have longer lifespans than males for biological reasons, and that there are quite a few reports that the lifespan of castrated males is significantly increased. If 17α-estradiol (or estradiol valerate perhaps) does not extend human male lifespan, I would have to believe there is some other similar route that likely does, and we just have to find the best way to go about pursuing it.

Option two is, in my opinion, moderately plausible. It could the case that when we do have groups that supplement estradiol, the dosage taken is nowhere near sufficient for a noticeable longevity improvement, and that if we would simply increase it by some factor, longevity benefits would become apparent. There does seem to be a dose-dependent relationship for the longevity benefits in mice, and it may be possible that estrogen receptor alpha simply isn’t being agonized nearly enough. This may depend on the type of estrogen and route of administration used, as well as other drugs that may be taken (for example, most male to female transgender individuals take an anti-androgen as well as an estrogen, and this could potentially ablate benefits). My personal conjecture would be that estrogen monotherapy via injections would have the best probability of a longevity benefit for those assigned male at birth, although modulating or combining this with SERMs may also be of interest, although much more experimental and difficult to get right (I may add more to this later as this is a pretty interesting avenue to me for multiple reasons).

As for option three, it may seem difficult at first glance to think that millions of male to female transgender individuals are all currently supplementing a substance that may increase their lifespan by 5-20%, but yet none of us (or them) have noticed this yet. However, there are no preventative reasons for why this couldn’t be the case, nor statistical evidence against this possibility. It could even be that suppressing testosterone and activating estrogen receptor alpha are additive in nature, and we end up with a particularly impressive lifespan extension effect from conventional feminizing HRT.

Although I obviously cannot be sure of any specifics, I do think there is likely some hormonal intervention that should significantly increase male (assigned at birth) human lifespan, but that we just may need another decade or two to get the optimal intervention figured out properly. It would be great to have substances like 17α-estradiol in human trials already, as the potential ROI for successful longevity interventions is massive both in terms of billions of additional QALYs and trillions of dollars saved in healthcare expenditure.

In conclusion, 17α-estradiol might notably extend human lifespan for those assigned male at birth. There are many potential mechanisms of action that could cause this, with the most interesting one perhaps being activation of the mTOR and AMPK pathways, resulting in more ‘feminine’ DNA methylation. This longevity benefit, if it exists, may apply to many male to female transgender individuals, or could also be weaker or stronger for various reasons, such as due to the common usage of anti-androgens. If this longevity benefit does not apply to these groups, there may be alternative hormonal interventions that work instead, such as supplementing 17α-estradiol directly, using a SERM with a strong binding affinity in the right areas, or other modifications to the HPG axis that reduce some potential negative longevity effects of testosterone.

Disclaimer: I’m a random person on the Internet and none of this is medical advice. I’d like to rewrite and expand on the potential mechanisms of actions in this post and talk a bit more about what I do myself in this area some time too. Feel free to mention any corrections or comments to me (see: About page).

Cynical Answers To NFT FAQs

Q: What is an NFT?

A: NFTs are a novel method for some people to make boatloads of money off of others, and in doing so create an entire new ecosystem that primarily uses misinformation to justify its own existence in order to perpetuate profiteering from those at the top.

As one would expect, many previous methods with this MO have existed, including within the cryptocurrency ecosystem, such as ICOs. However, most ICOs ended up being illegal as they not only involved selling unregistered securities to non-accredited investors, but also involved a lot of fraud and deception. NFTs solve this debacle by having a significantly lower legal risk, as they’re unlikely to be considered securities (since I wrote this, many people have come up with wonderful ideas on how to turn them into securities, so this can be considered false for many projects. Regardless of this, there’s enough other laws about fraud being broken that it is often irrelevant.)

Technically, an NFT is an entry (digital file) on a blockchain (large sequence of blocks made up of data and transactions) that is unique, and thus not fungible (interchangeable) with any other token or asset.

Q: How does an NFT put art on the blockchain?

A: It doesn’t. The reason for this is simple: the blockchain is too inefficient to store large amounts of data on; storing as much as a 4MB image on the Ethereum blockchain would currently cost around $72,000, as it needs to be stored on every copy of the blockchain in existence (The math for this is (10^9 / 10*18) * 1000^2*4 * 68 * 150 * 1800, where the constants are the following: wei per gwei, wei per eth, bytes in 4MB, gas per byte, gwei per gas, usd per eth).

Q: If an NFT is not on the blockchain, then where is it?

On a web server somewhere, just like everything else on the Internet. Specifically, the blockchain may have a link to media content, which in the best case would be an ipfs link (which is still sitting on one or multiple computers somewhere, and generally accessed only through centralized gateways), and in the worst case is an http(s) link. Neither of these are guaranteed to remain in existence forever, but at least ipfs (among some other decentralized solutions which are still relatively newer) can be partially decentralized, replicated more easily, and verified more easily. As the blockchain is public, all files are generally public as well. This not only means that the content pointed to by the NFT may not stay up, but also that it could be replaced with anything else, as recently pointed out by Moxie.

Q: What about other NFT information like traits or NFTs listed for sale on websites? That’s the blockchain, right?

Nope. Again, due to the blockchain being prohibitively expensive to store information on, even NFT traits (identifying characteristics/labels for the given token) are generally not stored on the blockchain, but are instead provided via a JSON api, just like the rest of the Internet uses.

Although NFTs are intended to be minted on the blockchain in order to exist, the cost of this started to get too high as fees increased, so now popular websites that users use to create NFTs have a ‘gasless’ minting method, where no blockchain transaction occurs until someone purchases the asset on the website, thus the blockchain is yet again bypassed and a centralized entity is used instead. If you analyze the technical makeup of many popular cryptocurrency projects, this is an extremely common theme; most cryptocurrency blockchains are very expensive, redundant, inefficient, and slow; so centralized systems are used in their place anywhere that users are not directly paying attention to.

In fact, it’s often much worse than this! As pointed out by Moxie, centralized companies like Opensea can remove NFTs from their platform at their own discretion, and ‘decentralized’ extensions such as metamask just query the Opensea API! Working with blockchains is very expensive and difficult and tedious (for a good reason – decentralization is hard and is often worth this effort!), so this is a very common pattern (we are certainly glad actors like Etherscan seem to be impartial, because almost all chain information comes from companies like this rather than from anyone reading the blockchain data themselves!)

Q: If transactions on the blockchain are so expensive, how are users using Ethereum to make cheap and instant transactions?

A: They aren’t, at least not right now. Currently the cost to transfer an Ethereum ERC20 token is $22 and the cost to trade a token with Uniswap is $65 (this seems to have only increased since writing this and constantly changes, so this section will often be out of date). A regular transaction can still be performed for around $6, although this can of course increase arbitrarily according to the market. This price may decrease at some point, but you also never know when the market will increase it drastically, potentially even making ether you own worthless (for example, if the fee to send eth is $6 and you only have $5 of eth in your address, you are out of luck). It is worth knowing there are other solutions (sometimes called L2 / ‘Layer 2’ systems) that are working to improve this on many major blockchains such as Polygon on Ethereum.

Q: How do I receive ownership and the rights to the art I purchase as an NFT?

A: You don’t. As far as ownership goes, there is nothing but a digital signature by an Ethereum address you have the private key to, which is placed on a contract that has a link inside of it of something you happen to like. Anyone can see the link and view the file. Additionally, there is nothing legally binding about this transaction, and there is no guarantee you will have the IP rights to whatever it is you spent your money on. Many popular NFT projects specifically have legal disclaimers telling you that not only do you not own the IP, but they (the creator) does, and you are unable to modify it without their permission.

Q: How can I ensure the original artist is the person selling the NFT?

A: You can’t. Anyone can create an NFT that has any link to any file in it, and there is nothing preventing this from being published on the blockchain by anyone.

As you would expect, there are many instances of users selling art that they did not create. In addition to art being stolen and sold by someone unrelated, resources such as machine learning models and art tools have been used to create valuable NFTs, with the original programmers not only left uncompensated, but un-credited entirely. But at least some random person got $10,000 for taking credit.

Q: Why has the popularity of NFTs been increasing so much?

A: Because people are making easy money with them. Similar to cryptocurrencies, every person that owns them has a vested interest in hyping them up to others in order to profit. The ecosystem as a whole uses many techniques in order to increase its own virality, including stories about how Everyone Is Getting Super Rich Super Quickly Doing Basically Nothing Except For You, significant hype both from excited individuals and from extensive paid shilling campaigns from those that are set to profit from them, and new technical jargon like “Decentralized Ethereum non-fundigle tokens with sidechain and parachain integration using ERC721+ERC1151”.

Q: How can I verify that an NFT purchase was legitimate?

A: You cannot. Although the transaction is on the blockchain and you can verify that it occurred, you do not know who the addresses involved in the transaction belong to. This enables one to create NFTs and then buy them from themselves using different addresses that they own in order to give the appearance that they are valuable and in high demand, effectively painting the tape with the hope that someone else (who, unfortunately, doesn’t understand this is occurring), will then will pay a large amount for something no one else actually wanted. For example, the recent NFT purchase for $69 million which garnered significant media coverage was even publicly known to have been someone that already had a prior business relationship with the seller. Regardless, it seemed to have made a good enough story to make it to just about every ‘news’ website – which was exactly the intention of this purchase

Q: Why do you hate cryptocurrencies or Ethereum so much? You must be a fiat supporter!

A: I don’t hate cryptocurrencies at all; I actually love the concept of many of them and think ideas like Bitcoin and Ethereum have been revolutionary. I own Bitcoin, Ethereum, and Polkadot, and enjoy using them. I do kind of support fiat, however, so you might have me there; my need to pay bills and taxes is unfortunately not circumventable right now.

Q: How can I learn more about how NFTs are marketed?

A: This video is my favorite single resource to show someone who would like to learn how they can get rich quick by copying the well-known methodologies of the pros. This video is not about cryptocurrencies, but you’ll find that the common tactics of market manipulation work just as well in cryptocurrency markets as they do in traditional finance.

There’s many tactics commonly used that are not mentioned in this video as well, including purchasing social media interaction (Twitter followers, retweets, Discord server members, Reddit posts, Reddit upvotes, many others), having multiple levels of insiders who get stakes in projects before anyone else does and then consequently hype them up, purchasing press releases and news article about projects encouraging positive price action with forward-looking statements, wash trading and painting the tape in order to increase the perceived price and price increase of items (things like this may even be outsourced or fully automated. There’s money to be made, after all), copying art and code from others in order to quickly seek a profit with even less original work, and in general many other forms of fraud, of which the goal is to convince users that they will make money when engaging in actions that have specifically been designed to enrich parties other than themselves, often where said other parties are 1) significantly more well-versed in the workings of cryptocurrencies and the markets they are operating in, as well as 2) acquired their NFTs/coins/tokens/DefinitelyNotSecurities at significantly lower prices far before most other users were able to, and thus stand to gain asymmetrically better risk and reward for their activities, which generally consist of marketing in every shape and form imaginable, no matter how annoying or fraudulent (hence NFTs being an inherently viral phenomenon – there is no better way to artificially induce a high R0 in a meme than to directly incentivize it via rewarding large profits to those who are the most effective at spreading it).

Comments on Firefox

This post is a summary of some of the things that I dislike about Mozilla and Firefox. Given how passionate I am about user rights, privacy, decentralization, FOSS, etc, sometimes these remarks surprise people. I still am grateful Mozilla exists, I still use some of their products, and there are many amazingly smart and good people there However, donations to Mozilla for the purpose of improving Firefox are ineffective and are better spent elsewhere, and I really wish they would stay focused on making better products.

Mozilla has a lot of money, most of which does not go towards Firefox

Mozilla constantly mentions that they are a nonprofit, encouraging you to donate to them to help Make The Internet A Better Place. While the Mozilla Foundation is legally classified as a nonprofit, their subsidiary, Mozilla Corporation, is not. Its revenue is around $450,000,000 per year, almost all of which comes from their contracts with Google (Yandex and Baidu as well). Google pays Mozilla half a billion dollars per year because Mozilla has contractually agreed to keep Google as their default search engine, and presumably this gives Google a net profit, as having more ad views and user information is very, very valuable (I heard an explanation that Google also wants to avoid antitrust issues, but I’m unsure of the veracity of that).

The CEO of Mozilla (both the foundation and the corporation), Mitchell Baker, had an annual income of over $3,000,000 (official source, cannot find 2020 document yet) or around $1,000/hour, which has managed to increase for several years in a row, despite metrics related to Mozilla’s primary product, their web browser, significantly decreasing (do note: her salary is technically from the for-profit, not the non-profit). In case you haven’t seen a report on browser market share in the last few years, Firefox currently has around 3-4% of the market share, with the highest estimates I can find being around 7-8%.

Yeah, it’s kinda bad

Most of Mozilla’s spending does not go to software development nor Firefox, but rather to administration, marketing, and similar expenses (this is true both for the non-profit and for-profit, but the non-profit’s information is publicly available). Checking their most recent form 990, there was $4.5M spent on grants to random universities and groups, $3M spent on management fees, $1.8M spent on travel fees, and $0.8M spent on conference fees, which combined is significantly more than what is spent on employee compensation. This means when you donate to Mozilla’s nonprofit, your money is more likely to be spent on universities, management, and travel than an employee’s compensation.

Although there are not many publicly available details about the specific spending of the for-profit section of Mozilla (which would be ~10x more), the distributions appear to be relateively similar from what sources I can find. One reason why Mozilla spends so much on marketing is because their products are generally not as good as their competitors’, and attempting to purchase your way to a larger userbase is an expensive and constant uphill battle.

Firefox does little to stop you from being tracked by Google

One of the most popular reasons to use Firefox instead of Chrome is a dislike for being tracked by Google. While it’s true that Google recieves more information from Chrome users than it does from Firefox users, the majority of information flow remains a constant, and Mozilla relies on a plethora of Google services for basic browser functionality. Here are some examples:

  • Firefox uses Google search by default and sends all queries/address bar typing to Google
  • Firefox uses Google’s safe browsing service for ‘unrecognized downloads’, sending Google the filename and url that you visited
  • Firefox uses Google services for basic APIs such as their location API, despite Mozilla having attempted its own implementation, which one may assume they’d use. As Firefox polls your OS for information to send to this API, this sends information such as your wifi-network or nearby phone towers in addition to your IP address and a biweekly-rotating Google client identifier.

Apart from Firefox relying heavily on Google’s services, unless you use extensive tracking/blocking addons, you’re being tracked everywhere you go to begin with, as the majority of websites use Google Analytics, Google APIs, Google fonts, Google ReCAPTCHA, among many others.

There is truth to Mozilla working on and implementing important privacy improvements in browsers, such as DNS over HTTPS, third party cookie blocking by default, tracker blocking, and so on. Some of these appear to be helpful, however are easily mitigated by other parties, while others are more questionable (for example, the implemention method of rolling out DoH by opting users into it, bypassing their network configuration preferences, and sending all DNS queries to a single company’s servers, was not optimal). As of firefox 86 on Feb 23rd 2021, Firefox appears to be attempting full per-site cookie isolation, which if successful and usable could be a great improvement here.

Firefox includes tracking, advertisements, and backdoors

Mozilla takes almost every chance it can to tell you how much they love your privacy, and for that reason the tracking and default features that are included in distributions of Firefox are pretty surprising (this does not mean Firefox is worse at this than other browsers!).

By default, firefox shares all of the following with Mozilla:

– the number of open tabs and windows, the number of websites visited, the number and type of addons installed, the length of your browser session, all interaction events with ‘Firefox features offered by Mozilla or our partners’, your device information, OS information, hardware information, and your IP address

  • “Firefox uses your IP address to suggest relevant content based on your country and state”
  • “When you choose to click on a Snippet link, we may receive data about the link you followed”
  • “Firefox sends basic information about unrecognized downloads to Google’s SafeBrowsing Service, including the filename and the URL it was downloaded from”

Some of this information is reasonable, such as crash reports and the base OS/Firefox version, but I still found this to be more than many may expect.

Firefox now comes with a feature called studies, which allows Mozilla to remotely install and run custom changes and featuresets to your browser without asking you. This is turned on by default, which is generally all that matters as almost no users go through every setting in software to turn things off manually. In the past Mozilla used their ability to remotely control browser installations to install an addon into users’ browsers that gave them cryptic messages which were intended to be advertising for a TV show. I don’t know why they thought that was a good idea, as it seemed to be almost unanimously agreed upon that it was a terrible idea, but it still happened. If you visit about:studies in Firefox you can see which studies you have/are currently participating in. I could not find any resource from Mozilla that lists all studies that they run, or anything remotely like this.

Firefox continually pushes sponsored and clickbait content into their products

Firefox comes with many features for sponsored content and advertisements, such as Sponsored Top Sites. The Mozilla page about this feature says they send ‘anonymized technical data’, which is hyperlinked to a near-empty Github repositry. Firefox partners with adMarketplace for this, which states “We may also receive technical information such as your approximate location, browser type, language settings, user agent, timestamp, cookie ID and IP addresses”, which is very dissimilar to what Mozilla says about this tracking, but perhaps they have a special agreement with Mozilla to opt their users out of this or something.

There is also Pocket, which comes with all distributions of Firefox and shows sponsored stories and other content ‘curated by our editors’ by default. I’m not even going to pretend this is decent. This is is a terible feature, and the last thing I want to see when I open my web browser is a bunch of advertisements for clickbait. I find it sad that mozilla says Pocket “Trades clickbait for quality content”, when the majority of content Pocket offers is complete trash designed to make you click and waste your time, including a lot of content that suggests that surveillance and censorship of the Internet is required to keep me informed and safe.

Please keep your clickbait out of my browser’s default home page, thank you

Fortunately several of the worst features of Firefox are easy to turn off, and some features that are even worse such as Ion, which literally just sends your browsing history to ‘researchers’, are disabled by default and must be opted into. I’m unsure why these features are included in Firefox to begin with, as I can’t imagine the small revenue stream they introduce is significant nor in Mozilla’s best interest.

Firefox is generally a slower browser than Chrome

If you’re a Firefox user, I suggested using Chromium just for a few minutes. I usually use forks of Firefox or Firefox ESR, but when I use Chromium I’m sometimes stunned at how much faster it is. Finding fair and recent browser benchmarks is difficult, but but most of which I’m able to find seem to confirm this, and testing local website rendering myself results in Chrome not just being slightly faster, but often 50-300% faster with its rendering, network requests, and javascript execution.

Using Chrome for a few minutes instantly allows me to understand why it has dominated Firefox in market share over the last decade. While it’s true that Google has many inherent advantages in promoting software, I think its performance, speed, and UX alone goes a long way in demonstrating why Firefox has fallen behind so far.

Mozilla has strange and contradictory ideological goals

When visiting the homepage mozilla.org, the first article that is shown to me is titled ‘We need more than deplatforming’ written by the CEO Mozilla, which implores us to do things like “Turn on by default the tools to amplify factual voices over disinformation”, kindly linking us to a NYT article that discusses how ‘authoritative sources’ such as the NYT and CNN should be prioritized over independent voices.

I don’t want to write much about politics for reasons that should be obvious, but attempting to solve the world’s problems via fact-checking is terrible naive and has no chance of working out well, and I’m amazed at how many large organizations seem to act like the solution to our society’s problems is for us to just ask a fact-checker what is true and what is false, and then hold hands and sing songs as our new utopia is formed.

Continuing to read blogposts from Mozilla (this is from their foundation’s website, I should add that they have some good technical blogposts in other locations) is a rather interesting endeavor as it becomes more and more apparent that Mozilla has large segments of their organization that don’t seem to have any clear goals, and just kind of write about random social and political things on the Internet and their opinions on it, sometimes throwing six-figure grants to random groups of students to make a game that no one ever plays to show us about how something is obviously bad, which I assume they thought was a better use of their money than hiring someone to work on Firefox (which 250 people were laid off from last year).

While Mozilla attempts to provide commentary on many important social issues, there appear to be many suggestions that go directly against their manifesto, which suggests that open expression and individuals freedoms should be prioritized. I respect the right for Mozilla to spend its funding on any social or political content it chooses, and I also think that many of the issues they dedicate time to are very important for our society and for the Internet, but I would rather their organization focuses on making a good web browser, because I would be much more excited about donating to them if my money went towards that.

I’m still glad Firefox exists

I’ve written some negative views about Firefox, but I’m still glad it exists. Making a perfect web browser is difficult, and trying to respect user privacy is difficult. I think Mozilla would be much better off if they were a product-focused company, and spent more money on technical innovation and additional engineers and innovators. For this reason I don’t think donating to Mozilla is a good choice, and as far as similar organizations go, prefer the EFF instead.

This article started to turn into a bit of a rant as I’ve continually found myself disappointed with decisions Mozilla has made, and I’m not surprised that their browser market share has decreased by almost 90% over time as a result. It’s easy to criticize, but difficult to build, so I do want to include this disclaimer to restate that I’m glad Firefox and Mozilla exist, and I wish the best for them and their browser, but I think their modern directions are distracting them from making products good enough to be widely used. I hope things improve, because it would be nice if more than one or two web browsers existed. In fact, I think that’s very important.

This Anime Does Not Exist

This Anime Does Not Exist was launched on January 19th 2021, showcasing almost a million images (n=900,000 at launch, now n=1,800,000 images), every single one of which was generated by Aydao‘s Stylegan2 model. For an in-depth write-up on the progression of the field of anime art generation as well as techniques, datasets, past attempts, and the machine learning details of the model used, please read Gwern’s post. This post is a more concise and visual post discussing the website itself, thisanimedoesnotexist.ai, including more images, videos, and statistics.

The Creativity Slider and Creativity Montage

Previous versions of ‘This X Does Not Exist’ (including anime faces, furry faces, and much more) featured a similar layout: an ‘infinite’ sliding grid of randomly-selected samples using the javascript library TXDNE.js, written by Obormot. Something more unique about Aydao’s model was that samples remained relatively stable, sometimes significantly increasing in beauty and detail as the ‘creativity’ value (see: psi) was increased all the way from 0.3 to 2.0. This led to the creation of the ‘creativity slider’ as well as linking every image to a page with a tiled layout showing every version of the image:

“The more entropy you give me, the more it makes me want to smile!”, 18 samples of image # X from creativity 0.3 to 2.0
One of the best and most common effects of higher creativity is better colorization, including shading, reflections, vibrancy, color diversity, and more
Another tile of seed 7087, showcasing stability with an increasingly interesting artistic style and details

Although the above two are among my favorite examples of creativity montages, many users found some other interesting things that increased creativity could do:

Sometimes it appeared as if ‘creativity’ was simply an alias for the increase in volume of a certain bodily area coupled with a proportional reduction of garment-covering surface area (seed 30700)
Sometimes increased creativity also leads to mitosis (seed 28499)
Sometimes you get… a lot of mitosis?

Selected artwork

The best way to demonstrate the stunning potential of this model is to show some of the samples that users have enjoyed the most:

The original thumbnail image for This Anime Does Not Exist: “Notice me, Onee-chan!”
Other earlier candidates considered for the website’s thumbnail (seeds: 1013, 4606, 8674, 8859)
Four popular images selected by Twitter users
Images from seeds 16736, 23655, 62731, and 93887
Images from seeds 84995, 50428, 10157, and 52476

Weirdest images

Incidentally, it also seems that users love sharing images that are not the most beautiful, but the weirdest as well. The below four were among the most popular images on the website during the first few hours, when there was only n=10,000 of each image:

A montage of the interesting and popular results from images 3313, 7820, 4437, and 3103. Sample 3103 would be a particularly good album cover for a death metal band.

Collapsed images

Sometimes the result collapses enough to lead to an image that, although pretty, does not at all resemble the ideal target:

interesting but unintended results from samples 0544, 31975, 3997, 0557

Writing

In many cases the model will produce writing which looks distinctly Japanese, however upon closer introspection, is not legible, with each character closely resembling distinct counterparts in Japnese scripts, however diverging just enough to produce confusion with a lingering feeling of otherworldlyness.

Although some characters are easily recognizable, many are not, and are nonetheless *usually* combined in incoherent manners

Videos and gifs

As it’s possible to produce any number of images from the model, we can also use these images to produce videos and animated gifs. The primary style of this is referred to as an interpolation video which is produced by iterating through the latent variables frame by frame, transitioning in between different samples seamlessly:

Interpolation video produced by arfa

Additionally, I decided to make a few videos that used a constant image seed, but modified only the creativity value instead (instructions on how I did this are here):

I chose this particularly wholesome sample to demonstrate a gif with creativity 0.3-2.0 and a frame difference of 0.01

Statistics and users

After the first day, the website had served over 100 million requests, using over 40TB of bandwidth:

First-day traffic statistics from CloudFlare’s Traffic Analytics

At launch, the largest contributor to traffic by country was the United States, followed by Russia, but over the next two days this quickly shifted to Japan.

A map showing which of Cloudflare’s data centers served the most DNS query results for the domain

As of writing this post (Jan 21st, 2021), the largest sources of traffic appear to be from Twitter, Hacker News, Gigazine, Hatena, and Himasoku.

Costs

Compare to similarly-trafficed websites, thisanimedoesnotexist.ai was relatively cheap, thanks to not requiring server-side code (serving only static content):

  • Domain ‘thisanimedoesnotexist.ai’: for two years from NameCheap: $110

  • Cloudflare premium, although not required, improves image load time significantly via caching with their global CDN: $20

  • Image generation: 1,800,000 images, 10,000 generated per hour with a batch size of 16 using a g4dn.xlarge instance, which has a single V100 GPU with 16GB of VRAM at $0.526 per hour on-demand: $95

  • “Accidentally” hosting the website from AWS for the first day, resulting in over 10TB of non-cached bandwidth charges: >$1,000 (via credits)
An image of what my desktop looks like while generating an additional million anime images
What the access log for nginx looks like while serving >1,000 anime images per second
Yet another example of why to not use AWS for high-bandwidth content: AWS has some of the most expensive bandwidth at $0.09 per GB (luckily this was paid for entirely with credits and the migration off of AWS was complete in less than a day to a more sustainable provider)

Conclusions and the future

All in all, this was a fun project to put together and host, and I’m glad that hundreds of thousands of people have visited it, discussed it, and enjoyed this novel style of artwork.

If you want to read in-depth about the ML behind this model and everything related to it, please read Gwern’s post.

Thank you to Aydao for relentlessly improving Stylegan2 and training on danbooru2019 until these results were achieved as well as releasing the model for anyone to use, Obormot for the base javascript library used, TXDNE.js, Gwern for producing high-quality writeups, releasing the danbooru datasets, and and other members of Tensorfork including Gwern, arfa, shawwn, Skyli0n, Cory li, and more.

The field of AI artwork, content generation, and anything related is moving very quickly and I expect these results to be surpassed before the year is over, possibly even within a few months.

If you have a technical background and are looking for an area to specialize in, I cannot emphasize the extent that I’d strongly suggest machine learning/artificial intelligence: it will have the largest impact, it will affect the most fields, it will help the most people, it will pay the most, and it will cause you to be surrounded by the best and smartest people you could hope for.

Thanks for reading and I hope you enjoyed the website! For more about myself, feel free to read my about page or see my Twitter.

Bonus: additional image compilations from Gwern

Quotes from The Everything Store – Jeff Bezos and the Age of Amazon

The Everything Store – Jeff Bezos and the Age of Amazon by Brad Stone is a book detailing some factors that led to the rise of Amazon as one of the largest corporate success stories of all time. I opened it expecting to skim through some parts, but ended up reading it in full in one sitting, and enjoyed it thoroughly. It left me with a strong sense of what makes Amazon, well, Amazon. And the best answer to that question is without a doubt, Bezos himself.

Rather than a full book review, I’m going to share some quotes from The Everything Store that stood out to me. One fun thing to note is that when this book was published in 2013, Amazon was ‘only’ a $150B company, but today is worth over 1.5 trillion. It’s a wonderful book and is worth buying if you want to read stories about Jeff Bezos’ extreme confidence in himself and his company as they overcome challenges one after another full-speed ahead, from local stores to Barnes and Noble to Ebay to Walmart and beyond. Below are some quotes that I particularly liked, only from the first fourth of the book. I was quoting the book more than anticipated, so I stopped this post early but will leave it up as an advertisement for the book.

Bezos is an excruciatingly prudent communicator for his own company.
He is sphinxlike with details of his plans, keeping thoughts and intentions
private, and he’s an enigma in the Seattle business community and in the
broader technology industry. He rarely speaks at conferences and gives
media interviews infrequently.

There is so much stuff that has yet to be invented.
There’s so much new that’s going to happen.
People don’t have any idea yet how impactful the Internet is going to
be and that this is still Day 1 in such a big way.
Jeff Bezos

Amazon’s internal customs are deeply idiosyncratic. PowerPoint decks
or slide presentations are never used in meetings. Instead, employees are
required to write six-page narratives laying out their points in prose, because
Bezos believes doing so fosters critical thinking. For each new product, they
craft their documents in the style of a press release. The goal is to frame a
proposed initiative in the way a customer might hear about it for the first
time. Each meeting begins with everyone silently reading the document, and
discussion commences afterward

“If you want to get to the truth about what makes us different, it’s this,”
Bezos says, veering into a familiar Jeffism: “We are genuinely customer-
centric, we are genuinely long-term oriented and we genuinely like to invent.
Most companies are not those things. They are focused on the competitor,
rather than the customer. They want to work on things that will pay
dividends in two or three years, and if they don’t work in two or three years
they will move on to something else. And they prefer to be close-followers
rather than inventors, because it’s safer. So if you want to capture the truth
about Amazon, that is why we are different. Very few companies have all of
those three elements.”

Bezos interpolated from this that Web activity overall had gone up that year by a factor of roughly 2,300—a 230,000 percent increase. “Things just don’t grow that fast,” Bezos later said. “It’s highly unusual, and that started me thinking, What kind of business plan might make sense in the context of that growth?”

Jackie Bezos suggested to her son that he run his new company at night or on the weekends. “No, things are changing fast,” Bezos told her. “I need to move quickly.”

Internet records show that during that time, they registered the Web domains Awake.com, Browse.com, and Bookmall.com. Bezos also briefly considered Aard.com, from a Dutch word, as a way to stake a claim at the top of most listings of websites, which at the time were arranged alphabetically.

Bezos and his wife grew fond of another possibility: Relentless.com. Friends suggested that it sounded a bit sinister. But something about it must have captivated Bezos: he registered the URL in September 1994, and he kept it. Type Relentless.com into the Web today and it takes you to Amazon.

They set up shop in the converted garage of Bezos’s house, an enclosed space without insulation and with a large, black potbellied stove at its center. Bezos built the first two desks out of sixty-dollar blond-wood doors from Home Depot, an endeavor that later carried almost biblical significance at Amazon, like Noah building the ark.

During that time, the name Cadabra lived on, serving as a temporary placeholder. But in late October of 1994, Bezos pored through the A section of the dictionary and had an epiphany when he reached the word Amazon. Earth’s largest river; Earth’s largest bookstore.3 He walked into the garage one morning and informed his colleagues of the company’s new name. He gave the impression that he didn’t care to hear anyone’s opinion on it, and he registered the new URL on November 1, 1994. “This is not only the largest river in the world, it’s many times larger than the next biggest river. It blows all other rivers away,” Bezos said.

One early challenge was that the book distributors required retailers to order ten books at a time. Amazon didn’t yet have that kind of sales volume, and Bezos later enjoyed telling the story of how he got around it. “We found a loophole,” he said. “Their systems were programmed in such a way that you didn’t have to receive ten books, you only had to order ten books. So we found an obscure book about lichens that they had in their system but was out of stock. We began ordering the one book we wanted and nine copies of the lichen book. They would ship out the book we needed and a note that said, ‘Sorry, but we’re out of the lichen book.’

A week after the launch, Jerry Yang and David Filo, Stanford graduate students, wrote them an e-mail and asked if they would like to be featured on a site called Yahoo that listed cool things on the Web. At that time, Yahoo was one of the most highly trafficked sites on the Web and the default home page for many of the Internet’s earliest users.

In the meetings, Bezos presented what was, at best, an ambiguous picture of Amazon’s future. At the time, it had about $139,000 in assets, $69,000 of which was in cash. The company had lost $52,000 in 1994 and was on track to lose another $300,000 that year. Against that meager start, Bezos would tell investors he projected $74 million in sales by 2000 if things went moderately well, and $114 million in sales if they went much better than expected. (Actual net sales in 2000: $1.64 billion.)

Bezos later told the online journal of the Wharton School, “We got the normal comments from well-meaning people who basically didn’t believe the business plan; they just didn’t think it would work.”11 Among the concerns was this prediction: “If you’re successful, you’re going to need a warehouse the size of the Library of Congress,” one investor told him.

When his goals did slip out, they were improbably grandiose. Though the startup’s focus was clearly on books, Davis recalls Bezos saying he wanted to build “the next Sears,” a lasting company that was a major force in retail. Lovejoy, a kayaking enthusiast, remembers Bezos telling him that he envisioned a day when the site would sell not only books about kayaks but kayaks themselves, subscriptions to kayaking magazines, and reservations for kayaking trips—everything related to the sport. “I thought he was a little bit crazy,” says Lovejoy.

The IPO process was painful in another way: During the seven-week SEC-mandated “quiet period,” Bezos was not permitted to talk to the press. “I can’t believe we have to delay our business by seven years,” he complained, equating weeks to years because he believed that the Internet was evolving at such an accelerated rate. Staying out of the press soon became even more difficult. Three days before Amazon’s IPO, Barnes & Noble filed a lawsuit against Amazon in federal court alleging that Amazon was falsely advertising itself to be the Earth’s Largest Bookstore. Riggio was appropriately worried about Amazon, but with the lawsuit he ended up giving his smaller competitor more attention. Later that month, the Riggios unveiled their own website, and many seemed ready to see Amazon crushed. The CEO of Forrester Research, a widely followed technology research firm, issued a report in which he called the company “Amazon.Toast.”


It was a distilled version of the dissatisfaction felt by many early Amazon employees. With his convincing gospel, Bezos had persuaded them all to have faith, and they were richly rewarded as a result. Then the steely-eyed founder replaced them with a new and more experienced group of believers. Watching the company move on without them gave these employees a gnawing sensation, as if their child had left home and moved in with another family. But in the end, as Bezos made abundantly clear to Shel Kaphan,family. But in the end, as Bezos made abundantly clear to Shel Kaphan, Amazon had only one true parent.

“You seem like a really nice guy, so don’t take this the wrong way, but you really need to sell to Barnes and Noble and get out now,” one student bluntly informed Bezos. Brian Birtwistle, a student in the class, recalls that Bezos was humble and circumspect. “You may be right,” Amazon’s founder told the students. “But I think you might be underestimating the degree to which established brick-and-mortar business, or any company that might be used to doing things a certain way, will find it hard to be nimble or to focus attention on a new channel. I guess we’ll see.”

“There will be a proliferation of companies in this space and most will die. There will be only a few enduring brands, and we will be one of them.”

During that time, no one placed bigger, bolder bets on the Internet than Jeff Bezos. Bezos believed more than anyone that the Web would change the landscape for companies and customers, so he sprinted ahead without the least hesitation. “I think our company is undervalued” became another oft- repeated Jeffism. “The world just doesn’t understand what Amazon is going to be.”

As the company grew, Bezos offered another sign that his ambitions were larger than anyone had suspected. He started hiring more Walmart executives.

Around that time, Wright showed Bezos the blueprints for a new warehouse in Fernley, Nevada, thirty miles east of Reno. The founder’s eyes lit up. “This is beautiful, Jimmy,” Bezos said. Wright asked who he needed to show the plans to and what kind of return on investment he would have to demonstrate. “Don’t worry about that,” Bezos said. “Just get it built.” “Don’t I have to get approval to do this?” Wright asked. “You just did,” Bezos said. Over the next year, Wright went on a wild $300 million spending spree.

“Walmart did not even have Internet in the building back then,” says Kerry Morris, a product buyer who moved from Walmart to Amazon. “We weren’t online. We weren’t e-mailing. None of us even knew what he meant by online retail.”

The venture capitalists backing eBay asked around and heard that one did not work with Jeff Bezos; one worked for him.

Bezos went skiing in Aspen that winter with Cook and Doerr and finally told them what was coming. “He said, ‘We’re going to win, so you probably want to consider whether to stay on the eBay board,’ ” says Cook. “He thought it would be the only natural outcome.”

If you liked these quotes, consider reading the full copy (perhaps even buying it from Amazon), it’s definitely a nice read about an amazing company and individual.