Isabel Dahlgren

AI for mathematics

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 29 Mar 2026 00:00:00 +0000

Over the last few months, AIs have helped solve several open problems within mathematics. This February, Epoch added a set of open problems within FrontierMath, their benchmark for mathematical research abilities. One of these problems – a Ramsey-style problem on hypergraphs – was recently solved by AIs: researchers creamed out solutions from Gemini 3.1 Pro, GPT-5.4 (xhigh), and Opus 4.6 (max). And by now (late March 2026), around 50 of the odd Erdös problems have been solved with the aid of AI tools. Amazingly, some problems were largely solved autonomously, without any human in the loop.

State of the art #

AIs have a number of strengths relative to humans. They excel at cracking competition-style problems – self-contained problems which involve a small set of techniques – and have done so for years. They’re also good at analysing large sets of data and generating hypotheses. For example, Gauss made long tables of $\int_2^x (\log t)^{-1} \ dt$ before conjecturing the Prime Number Theorem; such tasks seem amenable to AI automation. AIs are also good at exhaustive case analysis. Some results, such as the Four Colour Theorem, are difficult to prove using other approaches.

However, there are still areas where humans seem to outperform machines. While I’ve only used ordinary LLMs, rather than dedicated research tools like Prism, I’ll share some of my experiences.

Ordinary LLMs sometimes struggle to produce rigorous arguments, arguments where every step needs to be correct. The terminology ‘AI slop’ is fitting: unless they’re coupled with a proof verification engine, LLMs can be sloppy. For niche topics, I also find LLMs to be frustratingly inaccurate. LLMs often flood the chat with irrelevant information; good supervisors have an uncanny ability to focus on what matters. Experts give you the right reference, figure out the actual crux, ask the right questions.

This said, I imagine these weaknesses – the lack of rigour and general research taste – seem fixable with the right scaffolding. Overall, it seems quite plausible to me (≥ 60%) that AIs could automate most aspects of mathematical research within 10-20 years.

The future of mathematics #

Regardless of whether full automation is possible, it’s worth reflecting on how AI will transform the field of mathematics.

Terry Tao helped co-found the Foundation for Science and AI Research (SAIR) with the partial aim of exploring this question, and he has many interesting ideas on this topic¹.

First, as Terry Tao points out in the Dwarkesh interview, the use of AIs shifts the bottleneck to verifying arguments rather than coming up with ideas². Hence, I suspect proof verification software like Lean will play a larger role in the future of mathematics. In the most extreme scenario (we like the extremal principle, no?), mathematical research might reduce to operating AIs: have AIs generate 1 000 hypotheses, make other AIs double-check the reasoning, then employ your paper-writing AIs, iterate.

Second, the ease of generating new ideas significantly lowers the entry barrier for doing advanced mathematics: if you can obtain a proof from an LLM and have another AI system check it, you’ve effectively produced a proof. You might prove something without understanding what you’re doing³.

Conclusions #

Needless to say, AIs will play an important role in the future of mathematics, though it’s unclear how things will play out exactly.

I find it worth monitoring AI progress in mathematics for several reasons. Compared with more practical sciences like biology or chemistry, maths lends itself more easily to AI-driven progress. The nature of the subject, with results being either true or false, also enables efficient AI training; softer subjects give less RL signal. Moreover, using AIs to solve mathematical problems is also a problem that appeals to AI developers – many of whom are nerd-sniped maths graduates – so there’s a lot of effort going into AI for maths. In some sense, the ‘AI for science’ movement seems upper-bounded by the ability of AIs to do maths research.

In this post, he describes his use of AlphaEvolve to establish an integral bound. He writes: ‘Quite possibly AI tools would also have been able to assist with these steps [some intermediary bounds], but they were not necessary here; their main value for me was in quickly confirming that the approach I had in mind was numerically plausible, and in recognizing the right technique to solve one part of the toy problem I had isolated.’ ↩︎
To some extent, this is already the case – the review process in mathematics is notoriously slow. ↩︎
You could prove results without insight before even LLMs, thanks to software like Lean. Case in point: the liquid tensor experiment. However, AIs facilitate the generation of such proofs. ↩︎

Anki your life

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 22 Mar 2026 00:00:00 +0000

I’m rather distracted, forgetful and impulsive – just as most humans. People use different gadgets to focus on what matters: bullet journals, to do-lists and pocket-sized notebooks. I’m currently exploring an alternative: Anki.

Anki has many use cases beyond cramming for exams. We use our memory all the time, and whatever is memory-dependent is potentially ankifiable. I’ve been experimenting with ways to use Anki for procedural knowledge, and for the formation of mental habits specifically.

Examples #

Here are some examples of how others have used Anki to ‘develop a certain mentality’. I know this is a pretty vague goal, but I hope the examples will clarify.

To build a more accurate world model, you could e.g. use Anki to memorise geography facts. My friend Erik leads a group ankifying statistics from Our World in Data. If you want to make Fermi estimates on the fly, you need some data stored in your head.

Similarly, an aspiring superforecaster might memorise forecasting tips and tricks. In Superforecasting, Philip Tetlock characterises the forecasting methods of superforecasters:

In their methods of forecasting they [superforecasters] tend to be:

Pragmatic: Not wedded to any idea or agenda

Analytical: Capable of stepping back from the tip-of-your-nose perspective and considering other views

Dragon-eyed: Value diverse views and synthesize them into their own

Probabilistic: Judge using many grades of maybe

Thoughtful updaters: When facts change, they change their minds

Good intuitive psychologists: Aware of the value of checking thinking for cognitive and emotional biases

One way of translating this to Anki cards is by having a top-level card with front ‘Methods of superforecasters’ and back ‘PAD ProT Go’ (some acronym which is memorable to me), possibly adding cards decoding the acronym (e.g. front: ‘PAD Prot Go - first P’; back: ‘Pragmatic’).

Apparently people (presumably Christians) have ankified parts of the Bible too, as a way to build character.

Why would this work? #

I find Anki quite effective for building mental habits. Here’s my pop psych theory as to why.

Anki review sessions serve as periodic reminders. For example, suppose I want to make more data-driven judgements. Reviewing my Anki deck becomes like an affirmation of faith: each time I open the app, it’s like saying: ‘I’m doing this so I can make decisions grounded in actual data’. I find spaced repetition practise pretty boring, and I guess my subconscious motivational talk helps reinforce the habit.

And when it comes to adopting a certain mentality – being data-driven, making accurate forecasts or, well, being a good Christian – the main bottleneck might be forgetfulness. If only it occurred to you to compare against that baseline, to give probabilistic intervals rather than yes-no answers, or to think of the fifth commandment…

Maybe our schooling system is also supporting evidence that Anki can be used to build mental habits. I assume the ultimate goal of education is to instill certain mental habits. Memorisation is a key part of the humanist education program, tried and tested since the 1500s.

Conclusions #

Erik’s project is called ‘Anki The World’ – I suppose a natural summary of this post is ‘Anki Your Life’. Both students and non-students have a lot to gain from building a consistent spaced repetition habit.

A semester paper on harmonic analysis

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 15 Mar 2026 00:00:00 +0000

Can you recover a function $f$ from its absolute value $|f|$? Recovering $f$ exactly is impossible: if $\lambda \neq 1$ is a complex scalar with modulus $1$, then $|f| = |\lambda f|$ although $f \neq \lambda f$. Thus, one rather asks whether $f$ can be recovered up to multiplication by a unimodular scalar. If this is possible for arbitrary $f$ in a subspace $E \subset L^2(\mathbb R)$, we say $E$ does phase retrieval.

Further, we can ask whether the recovery can be done in a controlled way – stably. Given a subspace $E \subset L^2(\mathbb{R})$ and $f, g \in E$, is there a constant $C > 0$ such that

$$\inf_{|\lambda| = 1} \|f - \lambda g\|_2 \le C \||f| - |g|\|_2?$$

If so, we say $E$ does stable phase retrieval.

A cute question… #

The question of stable phase retrieval is easy to state but hard to answer, much like combinatorics questions. In fact, I wrote a 40-page semester paper about aspects of this question; stable phase retrieval a highly active research field within harmonic analysis. This blog post is intended as a mathematical apéritif, so you know which bits to read.

The semester paper is split in two parts: Part I surveys prior work; Part II is a modest attempt to extend prior work. We drop assumptions, simplify arguments and modify proofs for other problems.

My co-supervisor, Dr. Mitchell Taylor, who authored the 2022 paper on the first infinite-dimensional subspaces of complex-valued $L^2(\mathbb{R})$ doing stable phase retrieval, shared ideas on the extensions presented in Part I. These are the ideas developed in Part II.

I would like to thank both my supervisors, Dr. Mitchell Taylor and Prof. Dr. Alessio Figalli for making this project possible. I also want to express my sincere gratitude to Dr. Taylor for valuable discussions – both on the mathematical content and on the research process itself.

Part I #

Part I kicks off with some preliminaries from Fourier series and Banach lattices. Lacunary Fourier series are Fourier series with sparsity conditions on the frequencies appearing in the Fourier expansion. Apparently you can say a lot about their $L^p([0, 1])$ norms – for instance, they’re all equivalent. The statement of Nazarov’s inequality, which concerns another kind of ‘sparse’ Fourier series, was nowhere to be found except in his doctoral thesis written in Russian, and is another highlight from this subsection. We also develop some Banach lattice theory. Banach lattices are Banach spaces with a lattice structure which makes them the ‘right’ setting for studying stable phase retrieval.

Then we outline the problem of stable phase retrieval, supplying many examples and non-examples of subspaces doing stable phase retrieval. My favourite example of a subspace failing stable phase retrieval is the linear span of characteristic functions on $[0, 1]$ and $[2, 3]$; in my notation, that’s $\langle 1_{[0, 1]}, 1_{[2, 3]}\rangle$. To see why this subspace fails stable phase retrieval, notice that $u := 1_{[0, 1]} + 1_{[2, 3]}$ and $v := 1_{[0, 1]} - 1_{[2, 3]}$ have the same modulus, though there’s no $\lambda$ on the unit circle such that $u = \lambda v$.

We then proceed to discuss some striking results about stable phase retrieval in Banach lattices, justifying the earlier slogan about Banach lattices being good for phase retrieval. Just to wet your appetite: subspaces of Banach lattices doing stable phase retrieval admit an easy characterisation, and for finite-dimensional subspaces of Banach lattices, one can show that phase retrieval is automatically stable.

The final section of part one focuses on the first constructions of infinite-dimensional subspaces of $L^2(\mathbb{R})$ doing stable phase retrieval from 2022. We present two results, the first due to R. Calderbank, I. Daubechies, D. Freeman and N. Freeman (subspaces of real-valued $L^2(\mathbb{R})$); the second due to M. Christ, B. Pineau, T. Oikberg and M. Taylor (subspaces of complex-valued $L^2(\mathbb{R})$). The first construction involves a construction with orthonormal iid random variables and characteristic functions, while the second construction involves bases satisfying four well-chosen conditions.

Part II #

Part two is a modest attempt to extend the work on infinite-dimensional subspaces of $L^2(\mathbb R)$ doing stable phase retrieval.

First, we describe one way of weakening the conditions in the Christ-Pineau-Taylor result. One of the four conditions says that a set of functions should be orthogonal. However, because stable phase retrieval is an approximate condition (we want a ‘$\le C$’, not a ‘$=$’), and orthogonality is a condition of exact equality, we try relaxing the orthogonality to something like ‘almost orthogonality’. This leads one to consider so-called Riesz bases, which, intuitively, are ‘almost orthogonal’ sequences. We argue that the same proof goes through for Riesz bases too.

Second, we simplify an argument in the Christ-Pineau-Taylor paper, using a reduction suggested by a subsequent paper. Namely, one can show that orthogonal vectors constitute a kind of worst-case scenario for stable phase retrieval: if functions $f$ and $g$ break the stable phase retrieval inequality, then you can extract a pair of orthogonal vectors $f'$ and $g'$ violating the same inequality. So, by contraposition, it’s enough showing arbitrary orthogonal vectors from your subspace $E$ satisfy the stable phase retrieval inequality to conclude that the $E$ does stable phase retrieval as a whole.

Third, we adapt the proof of Calderbank-Daubechies-Freeman-Freeman to the setting of Pauli-stable phase retrieval. A subspace $E \subset L^2([0, 1])$ does Pauli-stable phase retrieval if, for $f, g \in E \subset L^2([0, 1])$, we have

$$\inf_{|\lambda| = 1} \|f - \lambda g\|_2 \le C (\||f| - |g|\|_2 + \||\hat f| - |\hat g|\|_{\ell^2}),$$

where the Fourier transform of an $L^2([0, 1])$ function is taken to be its Fourier series (which is in $\ell^2$, says Parseval).

In the report, we build subspaces from sine functions where the Fourier transforms play the same role as the characteristic functions in the original paper. Adapting their proof, then, leads to a result about infinite-dimensional subspaces of real-valued $L^2([0, 1])$ doing Pauli-stable phase retrieval.

The main difficulty in this proof was finding a workaround for independence – the original construction involves iid random variables, but our building blocks are sine functions. To have the functions in our subspace satisfy the same kinds of bounds, one can place sparsity conditions on the Fourier frequencies. And yes – this is where Nazarov’s inequality comes in.

This should be enough for an apéritif. I invite you to have a closer look at the report – the actual buffet. I had plenty of fun working on this project, and I hope this comes through in the full report.

Thinking outside the timeline

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 08 Mar 2026 00:00:00 +0000

How should we conceive of time? The question admits no easy answer, so different cultures have developed different time conceptions. The nature of time is something of a philosophical rabbit hole – I’ll try staying away from scholarly debates on time, instead focusing on the way ordinary people ‘feel’ time. What picture does the word ’time’ elicit?

In the West today, most people conceive of time as linear: time moves in a straight line, just like an arrow through space. Hence we represent time as lines – timelines.

In discourse on AI, believers in the transformative impact of AI (myself included) seem to assume a special conception of time. I think AI will lead to rapid technological progress and economic growth within the next few decades, and this could – though not necessarily – lead to accelerated social and technological progress.

Will MacAskill and Finn Moorehouse introduced the thought experiment of the accelerated decade to make the potential consequences of AI more visceral: imagine all scientific, technological, social and political progress from 1925-2025 occurring in the period 1935 – AI might catalyse the same kind of dizzying progress in the next decade. Implicit in their thought experiment is an event-driven conception of time: something happens around 2020-2040 (an intelligence explosion), which compresses the rest of the timeline.

My conception of time has influenced as series of big life decisions, so I wanted to inventory common time conceptions out there. As we shall see, our linear conception of time is a fairly recent phenomenon, and alternatives exist.

Some time conceptions #

For most of Western history, the notion of circular time dominated. In a circular time conception, our lives – and history more broadly – are viewed as a succession of repeated events, e.g. the four seasons or periods of war and peace. Plato introduced a circular time conception in Timaeus, linking time to planetary orbits¹. The idea of reincarnation, implicit in Eastern thought traditions, also suggests a circular view of time.

By the late 19th century, a linear time conception took hold. Emily Thomas gives four explanations to this shift in perspective.

These are all fun facts, so I’ll quickly rehash her views. First, there was the creation of time tables and timelines – if you want to visualise historical events, you can’t use circles – you have to use tables or lines. Second, there was Darwin’s theory of evolution (Darwin is everywhere). Third, she points to the development of chronophotography, a photographic technique which captures movements as a series of snapshots. Finally, physics: Charles Hinton developed a physical theory of time as a fourth dimension. From this perspective, then, time really was a line.

But other explanations are possible. You could also highlight the influence of Christianity. In the Bible, there’s a beginning and an end: Jenesis and Judgement Day. Time moves from beginning to end – time is linear.

As a special case of linear time, we can also speak of ‘binary time’², where you chop time into two distinct epochs. For example, some environmentalists distinguish between the pre- and post-anthropocentric era. Likewise, if you believe AGI could be huge, you might think in terms of pre- and post-AGI eras. Or, for a communist, the revolution marks a turning point in history.

Alternatives to linear and circular time exist. For example, Parmenides and Zeno viewed time as an illusion. Kant held a similar view, it seems. Or you could just be agnostic about your view of time. From Four Quartets:

Time present and time past Are both perhaps present in time future, And time future contained in time past.

Maybe one should just admit defeat?

Conclusions #

So much so for our overview of time conceptions. One conclusion emerges from these examples: your time conception is closely connected to your worldview. In a secularised society, a proxy question for ‘Do you believe in some God?’ might be ‘How do you view time?’ Sometimes you want to think outside the line.

: This said, there was a lot of confusion regarding time in Ancient Greece; Parmenides and Zeno, for example, viewed time as an illusion. ↩︎
There’s surely a better, more scholarly term for this. ↩︎

Is AI safety the new climate change?

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 01 Mar 2026 00:00:00 +0000

Ever since Eliezer Yudkowsky and Nick Bostrom first drew our attention to the risks of superintelligent AI, people have likened AI safety to climate change. Here, I’m referring to the overall goals of developing safe AI and transitioning to a net zero economy – both the technical and social dimensions.

Since the publication of Superintelligence (2014), AI safety has matured as a field. It’s no longer a fringe cause area of effective altruism or a topic confined to rationalist forums online. Whether for good or bad, AI safety is increasingly becoming like climate change in a number of ways.

Perhaps most strikingly, AI safety and climate change debates center around a point of no return. AI safety researchers speak of a technological singularity, the point where AIs start to recursively self-improve and we see AI progress taking off; climate change scientists speak of tipping points, temperature thresholds where additional warming triggers irreversible changes.

Similarly, within both AI and climate change, there are so-called ‘doomers’, i.e. people who think we’re bound for some kind of apocalypse. In AI, the archetypal doomer is Eliezer Yudkowsky, who seems to think AGI will inevitably kill everyone (or see the title of his book). Likewise, many AI safety researchers think catastrophic outcomes are highly likely in the absence of regulations. For instance, here’s Max Tegmark in an interview from November 2025¹:

…if we go ahead and continue having nothing like the FDA for AI, so people can legally just launch superintelligence and worry about getting sued later… yeah, I would think it’s definitely over 90% that we lose control.

Climate change activists like Greta Thunberg often employ similar doom-laden language. Someone has to sound the alarm.

And in both domains, the public has heard the alarm, as it were. By now, most people agree that we should build safe AI and solve global warming². As of September 2025, 50% of American adults were more concerned than excited about the use of AI in everyday life; in 2021, the same figure was 37%³.

Finally, I suspect that both AI safety and climate change are primarily policy problems. By this, I mean that the bottleneck is political willpower rather than a lack of technical solutions. Says Buck Shleregis of Redwood Research:

Five years ago I thought of misalignment risk from AIs as a really hard problem that you’d need some really galaxy-brained fundamental insights to resolve. Whereas now, to me the situation feels a lot more like we just really know a list of 40 things where, if you did them — none of which seem that hard — you’d probably be able to not have very much of your problem.

Moreover, Anthropic’s conflict with Pentagon highlight the dependence on policy-makers: AI safety is becoming a political issue. In climate change, it seems like we’re facing the same situation, with fixes being available but policy mess.

In conclusion, AI safety is becoming more mainstream – mainstream to the point of becoming more like a global coordination problem. And while there are a few differences⁴, discourse on AI safety and climate change bear an uncanny resemblance: the doomers speak of a doomsday, and the threat comes from big corps prioritising commercial interests over social considerations.

AI safety has its roots in the no-nonsense culture on LessWrong, and I’m hoping we can use this to handle AI safety better than climate change⁵.

Most people in the AI safety community hold more moderate views; see e.g. this article. Jonas Vollmer puts the probability of AI killing us and creating a world run by AI systems at 20%, while Buck Shleregis thinks an AI takeover is 40% likely. ↩︎
Interestingly, the recognition of the AI safety problem has led to an analogue of green-washing. ↩︎
For an overview of public opinion on AI, see this 80k podcast episode. ↩︎
See Sam Clarke’s discussion here. To me, the most striking difference is that the effects of transformative AI appear lesser understood. What will happen to the labour market? ↩︎
No, we’re not on track for our climate change goals. ↩︎

Should we all be doing evals?

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 22 Feb 2026 00:00:00 +0000

I try limiting my AI news intake, but I couldn’t help myself from following some of the news around OpenClaw and Moltbook. It began with the following tweet of Andrej Karpathy trickling down via a Slack channel:

Bought a new Mac mini to properly tinker with claws over the weekend. The apple store person told me they are selling like hotcakes and everyone is confused :)

And so went my weekend too.

During that weekend, as I saw more OpenClaw for myself, I began reflecting more on AI progress. In brief, I’m starting to internalise the idea that we’re in the midst of an intelligence explosion, and I think this means AI safety work should center around LLM evals.

Feeling the AGI #

Developers of ML models, whether in academia or industry, need something to aspire to – a holy grail of ML models. Right now, the holy grail is often called ‘artificial general intelligence’ (AGI).

Here’s the original definition of AGI from an eponymous research monograph from 2007¹:

What is meant by AGI is, loosely speaking, AI systems that possess a reasonable degree of self-understanding and autonomous self-control, and have the ability to solve a variety of complex problems in a variety of contexts, and to learn to solve new problems that they didn’t know about at the time of their creation.

The definition is vague, but you could make the case Claude Opus 4.6 satisfies these conditions.

The paper A Definition of AGI from October 2025 suggests defining AGI as ‘an AI that can match or exceed the cognitive versatility and proficiency of a well-educated adult’ and investigates the extent to which OpenAI models are AGI. We’ll stick with their definition for the rest of the post.

The authors assigned GPT-4 and GPT-5 AGI scores of 27% and 57%, respectively. Linearly interpolating AGI score progress based on release dates brings GPT-5.2, the latest OpenAI model, to around 60%. Regrettably, they don’t benchmark Gemini or Claude models, which I assume would perform even better.

Other benchmarks which test capabilities related to AGI, such as Arc-AGI-1, METR’s task horizon length, reveal rapid progress too. While unsaturated benchmarks still exist, most of them test beyond-AGI capabilities.

While I agree the expression ‘beyond-AGI capabilities’ sounds strange, remember that AGI isn’t the same as the holy grail of intelligence – for this article, we agreed that AGI was AI with the cognitive capabilities of a well-educated adult.

Depending on your personal disposition, then, you might feel the AGI. At least some of it.

Feeling the recursive self-improvement #

Just we can all feel some AGI, we can also feel some recursive self-improvement². Recursive self-improvement can happen at different levels: from training a deep network via backpropagation until AI systems running entire R&D cycles autonomously.

I view OpenClaw as a milestone in the development of recursively self-improving AI systems. While OpenClaw isn’t exactly R&D, it reveals the power of agent scaffolding. Weaker models can form the backbone of AI systems earning money, for example. And the more money you have, the more you can invest, and the more you earn. Maybe such money bots could be viewed as recursively self-improving AI systems.

As further evidence for self-improvement, Thibault Sottiaux from OpenAI reports that:

Codex now pretty much builds itself, with the help and supervision of a great team. The bottleneck has shifted to being how fast we can help and supervise the outcome.

Or take Dario’s forecast from March 2025 that 90% of code at Anthropic would be written by AI within 3-6 months.

But also feeling history #

The development of AI – just as any technology, really – follows a circular pattern³: booms, winters, booms. AI was established as a field of study in 1955 with the Dartmouth conference; the mid 1970s saw an AI winter, and was followed by a productive 1980s, with seminal work of Geoffrey Hinton and Judea Pearl. Hinton’s paper on backpropagation for neural networks was published in 1986, for example⁴. From the early 1990s until 2011, there was another winter. After that came AlexNet (Ilya Sutskever’s first major research achievement), AlphaGo, attention – most breakthroughs enabling modern LLMs, really. The last ten years correspond to the biggest AI boom to date, with the last four years – from the release of Chat-GPT 3.5 in November 2022 – arguably seeing a progress speedup.

If you think there’ll be an intelligence explosion or a technological singularity, it’s worth placing this decade in a broader historical context – look at a time scale with ticks for centuries rather than months. Then we appear to be in the midst of an intelligence explosion right now. We have some AGI – a good 57%, actually, if you follow the framework of the definition of AGI paper – and AI systems highly capable of recursive self-improvement. However, the best piece of evidence that we’re in an intelligence explosion now might well be the exponential law for task horizon length.

Crucially, we can start asking questions about what happened during the beginning of this intelligence explosion. Did the world become better, on balance?

Providing well-grounded answer to this question would require a book, or at least a series of blog posts, and it’s beyond the scope of this article. But we can draw important conclusions.

Conclusions #

We seem to be in the middle of an intelligence explosion, plausibly the final prophesied Intelligence Explosion or Singularity. We already have some degree of AGI and recursive self-improvement (if this isn’t an intelligence explosion, then what is?). This means we can, and should, ask more questions about the state of the world, so we can update our threat models. To what extent is GPT-5.2 scheming? How much fine-tuning does it take removing safety guardrails in open-weight models? Can existing frontier models perform economically valuable tasks?

The lack of AI regulation has led to a zoo of models to study. Over the last few years, AI safety has become less like natural philosophy and more like biology, involving lab work that can meaningfully inform the theory.

I very much welcome this development. Personally, I like benchmarking papers: whatever the benchmark results, that’s valuable information⁵. Theoretical alignment work – or even just attempts at developing new methods within empirical AI safety – seem riskier. Perhaps the best way to improve humanity’s long-term future comes down to doing API calls and doing honest write-ups of the results.

Anyway, if we want to coexist in harmony with the second species, let’s begin by understanding its evolutionary precursors. They’re already out there.

Thanks to Santeri Koivula for feedback on this post.

Reading the introduction felt much like reading a text from the 1300s – the book, freely available via Springer, is something of a historical document. ↩︎
I’m speaking of recursive self-improvement rather than ‘singularity’ because the term ‘singularity’ usually refers to an uncontrollable technological development. ↩︎
Ctrl-f ‘AI winter’ in this article, or just read Genius Makers. ↩︎
Backpropagation was originally introduced in 1969 by Arthur Bryson and Yu-Chi Ho, but in a more general context. ↩︎
Thanks to my mentor Zhijing for pointing this out. ↩︎

ML algorithms for life

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 15 Feb 2026 00:00:00 +0000

When planning, a good heuristic is to start visualising the desired outcome and recursively work out necessary steps: ‘I want to arrive at 8, so I need to leave at 7:30, so I need to finish breakfast at 7:10 – shoot, I need to get up at 6:30’. Here are further examples: ‘I want to meditate consistently, so I need to find an accountability mechanism, so…’ or ‘I want to reduce X-risk, so I need to find a job related to X-risk reduction, so…’

The basic algorithm is ‘I want X; to do X, I need to do Y; to do Y, I need to do Z…’ Backpropagation, no?

Computer science lingo is helpful for decision-making in real life, as highlighted in Algorithms to live by. In this post, let’s extend their ‘computer science $\to$ life situations’ mapping to the domain of ML.

Kinds of learning #

Supervised, unsupervised and reinforcement learning have close analogues in real life. Supervised learning learning is like acing tests, where the test result is your labelled data; unsupervised learning is like understanding deeper, non-examinable patterns. Reinforcement learning, where an autonomous agent learns optimal decisions through trial-and-error, is a good metaphor for early adulthood.

Similarly, it’s useful bearing in mind the distinction between test and validation sets. Ultimately, you want to minimise something like a generalisation error, the expected loss with data points drawn from a given distribution. This is different from your estimate of the generalisation error, the loss computed from the data points you happened to sample.

$$\mathbb E_{(X, Y) \in \mathcal P}[\ell(f(X), Y)] \neq \frac{1}{N} \sum_{i=1}^N \ell(f(x_i), y_i).$$

Don’t be too disheartened if the right-hand side is high.

Optimisation algorithms #

Stochastic gradient descent, too, is a good metaphor for how we learn. At a given moment in time, you figure out the best next step, take a step in that direction, and iterate. The gradient is computed across minibatches – you can’t feasibly use all data – so technically speaking, you’re probably doing minibatch SGD. The learning rate gives you the magnitude of the step. Terminology to describe loss landscapes (convexity, smooth, local/global minimum) is also useful, though it’s beyond the scope of this post.

As mentioned earlier, backpropagation also puts learning into perspective. Starting from the end goal, you work out necessary modifications to your parameters.

Further ML concepts #

So far, we’ve only discussed terminology relating to the basic ML pipeline. But there’s much more ML terminology allowing us to formulate problems from everyday life more precisely.

The bias-variance tradeoff captures the fundamental tension between low bias and high variance. A complex model is more expressive, but produces more variable outputs. Similarly, there’s the risk of overfitting. The conventional wisdom version: Occam’s razor.

We can also borrow jargon from label classification. Just as a model can be well-calibrated – combining the right signals in the right way – so can humans. If wisdom is about making the right decisions in light of available data, then the wise man is he who has high AUROCs across different classification problems. Another useful notion from ML classification is the max-margin solution: selecting the max-margin solution is like applying the precautionary principle.

Finally, principle component analysis also lends itself to personal real-world applications. For example, a good prompt might be: ‘What are the principal components here?’ And in this setting, the term ‘singular value’ should perhaps be understood literally.

Beyond human-brain analogues #

The field of ML has always used real-world analogues to drive progress – indeed, neural networks are modeled on human brains, and physical intuitions help with algorithm design. Conversely, thinking in ML terms might make us more clear-sighted. Maybe ordinary people can benefit from the field of ML in unexpected ways.

This post was inspired by a PEAKS workshop I attended this fall, led by Marius Wenk.

Growing older faster

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 08 Feb 2026 00:00:00 +0000

Every so often, I try remembering what Isabel was like one year ago. The more I cringe at younger me, the better – it means that I changed, though I’m often unable to explain how. Perhaps the cringe-meter captures something which traditional end-of-year reflection prompts don’t: my cringe-meter appears to measure the extent to which I’ve internalised new ideas, whether truisms about life or theoretical frameworks. Naturally, I’d like to speed up this learning process – in a sense, I’d like to grow older faster.

In my experience, this is largely a matter of getting exposed to new ideas. I’ll list a few classic ways to get exposed to new ideas and then offer some comments.

Examples #

Traveling might well be the definitive way to open one’s mind – young people have always gone abroad to do soul searching. In the 17th century, European aristocrats often undertook ‘grand tours’ around the age of 21; today, students are highly encouraged to do exchange programs.

Similarly, starting a new role is should expose you to new ideas. If you spend 40h per week at work, I bet something happens with you – whether you notice or not. Maybe it’s a change in attitude, values or language usage. For example, observe how quickly friends doing internships begin using business acronyms in everyday speech.

Studies can have the same effect. As a student, you spend much more than 40h per week at ‘work’, and you’re surrounded by like-minded people. The university experience often entails complete immersion in some geek culture.

Of course, the content you learn can also have a mind-opening effect. For example, doing pure mathematics makes you appreciate precision (and see the lack of it everywhere). However, I assume subjects like history, literature or economics – which concern our society more directly – have a bigger potential to change one’s outlook on everyday life.

Another class of ’learning experiences’ come from particularly intense experiences: getting married, watching your favourite movie for the first time or having a good madeleine. Traumas, negative intense experiences, also fall under into this class.

Comments #

What can be said about these examples?

The extent to which I internalise new ideas appears to be a function of both intensity and duration – it’s the integral that matters, as it were. Not all stays abroad are formative experiences: I need to be away for sufficiently long, and the environment needs to be sufficiently different. Likewise, for work and studies, I need an element of novelty and time to parse the new information.

But a major life change isn’t enough – there’s also the need for introspection. This might well be why traveling is so effective, at least for me: being abroad puts me in a reflective holiday state of mind.

Finally, observe that the common theme for the first three examples – traveling, work and studies – is novel encounters. My most meaningful learnings have come from conversations with people I trust deeply (whether dead or alive). Perhaps growing older faster is largely a matter of making new friends?

This blog post was inspired by a conversation with Elias Koschier about ‘going OOD’.

The power of network effects

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 01 Feb 2026 00:00:00 +0000

Returning to Sweden, I was struck by the differences in digital infrastructure between Switzerland and Sweden. In Zürich, you use WhatsApp, Twint and Luma; in Stockholm, Messenger, Swish and Facebook.

One of the great boons of coming to Switzerland was being free to delete Facebook. I’d reluctantly been using Facebook to be notified of events and for Facebook Marketplace, though I often found myself doomscrolling. I knew it was affecting my wellbeing negatively – I felt much better after deactivating my instagram two years prior¹ – so I tried lots of methods to minimise doomscrolling. Nothing worked (good job Zuck); getting rid of Facebook seemed like the simplest solution².

To be notified of events in Zürich, I relied on Luma, an event management platform, as well as newsletters and posters. Being a sucker for good UI, I became particularly fond of Luma. For messaging, I used WhatsApp and Signal, apps which aren’t nearly as buggy as Messenger.

Moving to Zürich meant I got to use better software in my day-to-day, and this had a surprisingly big positive impact on my life satisfaction. In fact, I suspect we’d be be much better off if we collectively decided to ditch bad software and use the apps everyone secretly wants to use anyway³. I also began reflecting on network effects more broadly.

When economists speak of ’network’ or ‘scaling effects’, they mean products which become more valuable as more people use them. A prime example is Facebook: people use Facebook because their friends use it, though, in reality, they might prefer alternative platforms. Markets with network effects are sometimes called ‘winner-takes-all markets’.

Let’s look at a few examples of positive and negative network effects, just to make things more concrete.

Amazon, digital payment platforms and Github are examples of good network effects. On Amazon, there’s a virtuous cycle: more buyers, more sellers, lower prices, more buyers, and so on – network effect, right there. Digital payment platforms like Twint, Wise or Swish only work when your friends have it, so also leverage network effects.

Social media, digital labor platforms and App Store are arguably examples of negative network effects. We could be using email, SMS and Discord to foster meaningful connections, but we’re stuck with general-purpose, ad-bloated platforms like Facebook and Instagram⁴. Network effects also lead to bad working conditions for platform workers⁵: food delivery drivers need to accept the conditions of digital labor platforms like Uber, Foodora or Wolt, often having to forego basic labor protections. Something similar happens on App Store: developers are forced, as it were, to sell their products on App Store and to accept Apple’s terms and conditions.

Unfortunately, such negative network effects are hard to disrupt. The standard way to address negative network effects seems to be through regulation, though good regulation is difficult.

Continental philosophers like statements of the form ‘X governs our lives’, where X could be ‘money’ (Marx), ‘power’ (Foucault) or ‘gravity’ (Weil). In the digital era, network effects can become ever stronger: the network can grow to include anyone with a smartphone.

For companies exploiting network effects: please, take care of us. For you and me in the network: today, network effects might govern our lives more than we think.

As soon as I could code a web-based photo portfolio. ↩︎
My friend Lukas developed an app to prevent him from doomscrolling; I wish it was around when I was still on social media. ↩︎
In fact, a 2-min Gemini-assisted Fermi estimate yields an increase in 2000 QALYs if Stockholmers switch from Facebook to Luma for event management. That corresponds to 25-80 lives saved – not bad. ↩︎
Discord works infinitely better than traditional social media, seriously. Any functionalities of Facebook relevant to social connection are also available on Discord. ↩︎
See this report from Human Rights Watch. ↩︎

My LLM stack

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 25 Jan 2026 00:00:00 +0000

You have a lot of power at your fingertips in early 2026, with highly capable AI cheaply available. Base models like GPT-5.2 and Claude Opus 4.5 have expert-level subject knowledge in most STEM fields, and they can autonomously complete tasks taking almost a full work day. And that’s just for base models. There are also powerful agent scaffolding tools for base models – take Claude Cowork, for example.

Whatever tasks an AI can solve with your oversight, you can solve too. Many ordinary people can solve the same kinds of problems as certified therapists, experienced architects or ML PhD students with reasonable effort.

It’s well worth learning how to use LLMs efficiently, then.

It feels like we’re told this all the time, and Demis Hassabis eloquently made the point at the seminar at the World Economic Forum this week. Because ’new year, new me’, I thought I’d listen.

Using LLMs efficiently #

Using LLMs efficiently seems to involve two things: first, identifying good use cases for AI; second, using AI to implement the solution.

Step two, execution, is normally just some kind of prompt engineering – a skill I’d expect to become less important in the future: producing user-friendly products is largely about eliminating the need for sophisticated prompt engineering. Step one is harder: it requires creativity and attention to failure modes in one’s everyday life.

To learn to use LLMs efficiently, I decided to catalogue the main ways in which I use LLMs. I’ll stress that I’m almost certainly using AI suboptimally and that these use cases are specific to my lifestyle, so my list shouldn’t be taken prescriptively.

For proper, well-curated lists of LLM use cases, I recommend browsing through the links listed by Gavin Leech in his post on LLM usage. Feel free to share your own favourite AI uses, e.g. via email.

Anyway, here goes.

Tried-and-tested use cases #

I have a low barrier for asking LLMs ‘dumb’ questions regarding individual proof steps. Similarly, I frequently use LLMs as sanity checkers, writing a sentence summarising my understanding of a topic and having an LLM give feedback.
LLMs are excellent for explaining terminology in plain, simple language. Two caveats, though: for looking up words in foreign languages, I strongly prefer ordinary dictionaries, like those of NE; I’m also skeptical of using LLMs for definitions.
I sometimes have LLMs lecture me on specific topics in mathematics. This is particularly useful when you’re self-studying rather than following a course with peers – e.g. when you’re doing a semester paper. Concretely, this looks something like dumping relevant papers, textbook chapters and lecture notes into NotebookLM and asking for a 3-page PDF summary focusing on key intuitions.
Of course, I rely heavily on coding agents, Claude Code being my favourite. Claude can handle most programming-related tasks today – even things like slurm management¹.
Routine email writing, like finding a time to meet.
Transcription of my favourite In Our Time episodes.
Just as Andy Masley, I also use AI for clearing ugh fields, like household chores. It’s not that I need AI for declaring taxes; however, it lowers the activation energy for the task. Or, to cite another example in this category, I used AI to repair my desktop lamp the other day.
Python scripts can save you a lot of time, and coding agents can whip up scripts in minutes. For example, I have scripts for creating Anki cards, for web scraping and for merging markdown files.

Experimental use cases #

I’m experimenting with using AIs to combat planning fallacy. I recently passed a project proposal through Claude, who categorically said that I was committing planning fallacy². After some back and forth with the LLM, I came away with a far better project proposal. To extend this point, perhaps I could have an AI coach me in some areas of my life?
I’m considering taking LLM input for some major life decisions, e.g. career choices.
Some people, like Kaj Sotala, use AI as aids for introspection. Maybe worth trying?

Conclusion #

As is evident from my examples, today’s AI is powerful enough to be useful in almost any domain of life. This year, I’ll continue exploring novel AI use cases, and I expect this to improve my quality of life in some respects. However, I’m also curious to learn whether AI can actually improve one’s day-to-day, or whether the main boons from AI might come from things like accelerated scientific progress.

It’s easy to see why programmers have notoriously short AGI timelines. ↩︎
Interestingly, I fed other AI models the same prompt (GPT, Gemini and Perplexity) – but I only received real pushback from Claude. And I want feedback that stings a bit. ↩︎

Building a CLI coding gym

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 18 Jan 2026 00:00:00 +0000

I attended a coding event this Wednesday, with participants hacking away on vibe coding projects and demoing their results by the end of the evening. My project was a CLI tool for developing tacit programming knowledge: Terminal Tutor. Code and docs are publicly available here, though this post is the real README.

I view Terminal Tutor as a coding gym – a way to keep your programming muscles from atrophying in the age of autonomous coding agents. And the programming muscles you’re maintaining aren’t about syntax ("plt.set_title or plt.title"?); instead, you’re maintaining, or even growing, precisely the muscles that matter in the age of vibe coding.

Defining tacit programming knowledge #

Specifically, Terminal Tutor aims to develop tacit knowledge, knowledge that’s hard to articulate in verbal instruction. To me it seems like tacit programming knowledge falls into two categories: consistently making the right design choices and terminal fluency. By terminal fluency, I mean instantly knowing which shell commands to execute, and with the correct syntax.

The best way to develop tacit knowledge is by working alongside an expert explaining their reasoning, i.e. through apprenticeship. Teaching skills matter a lot here, perhaps more than subject knowledge. But perhaps you don’t have the capacity to do a coding internship – the natural kind of coding apprenticeship – and in this case, LLMs might be useful. LLMs are both excellent tutors and excellent programmers, so it’s natural to ask whether one can use LLMs to develop tacit programming knowledge.

Developing a programmer’s eye for design is genuinely hard, even with LLMs. Developing terminal fluency – is also hard – the right bash commands are unknown unknowns. But LLMs can help develop terminal fluency, so this is why I choose to focus on terminal fluency.

Why terminal fluency still matters #

And to be clear, I believe you have good reason to develop terminal fluency, even in the age of autonomous coding agents.

There’s still a significant gap between a senior dev and a rookie with Claude Code Max – it’s not just a matter of coding the right thing rather than the wrong thing – experts also code the right thing the five times faster. And without being a senior dev myself, I think terminal skills play a crucial role here: terminal skills speed up whatever happens between LLM inference calls.

Also, knowing your terminal matters for safety: I’m more worried about someone mindlessly accepting a rm -rf / than about coding agents like Claude Code or Codex inserting backdoors in the code, like hard-coded solutions.

Last but not least, vibe coding is more fun when you can follow the commands executed; otherwise, it’s like watching a game of chess without knowing about common strategies. You want to vibe with the coding agent rather than being its sudo slave.

How Terminal Tutor works #

Terminal Tutor is a wrapper around Claude Code which has Claude explain all commands in detail – I like it when programming teachers explain every line of code, Karpathy-style – and log commands we’ve covered, so I can revisit them later. Under the hood, Terminal Tutor is just a CLAUDE.md file and some Python scripts to have Terminal Tutor remember mastered commands across repos.

Here’s the package; feel free to play around. I have ideas for future extensions (e.g. Anki integration via AnkiConnect), though more experienced programmers may have a better sense of what’s missing.

I’ve tested Terminal Tutor for my research projects over the last week, and it works surprisingly well: Terminal Tutor appears to be as good a coding agent as normal Claude. Though commands take slightly longer to run and I have to exert more effort, I learn more while coding. Ultimately, however, using Terminal Tutor is more fun – it feels good doing one’s programming reps.

Positive psychology for rationalists

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 11 Jan 2026 00:00:00 +0000

Books like Factfulness (2018) and Enlightenment now (2018) were very favourably received, partly because high-profile people were writing about the right things at the right time. More specifically, I suspect that these books largely owe their success to their feelgood component. Recall 2018: we had the Cambridge Analytica scandal, the Thunberg climate strikes, the murder of Jamal Khashoggi. Who wouldn’t have wanted to hear an grandpa-like professor saying that everything would be alright?

I recently discovered a wellbeing hack: reminding oneself of positive historical events, or positive psychology for rationalists. As I mentioned in an earlier post, I often think about the fall of the Berlin Wall – not just because history is important, but also because it makes me happy. Here are other pieces of historical trivia which similarly fill me with a sense of awe and wonder:

Convivencia happened, at least a bit: Around 800-1500 AD, Muslims, Jews and Christian communities peacefully coexisted in medieval Spain. This period is usually referred to as the ‘convivencia’ (Spanish for ’living together’ or ‘coexistence’). It wasn’t just that they didn’t kill people with other religious beliefs – there even appeared to be significant cultural exchange¹. After listening to the In Our Time episode about Muslim Spain, I came away feeling very optimistic about human nature.
Dissolution of Soviet Union: One could argue that the USSR dissolved without bloodshed: the USSR didn’t end after a treaty following a bloody third world war. Though you might object that the Cold War wasn’t bloodless, we could have had a nuclear war. Gorbachev was rather friendly, at least in comparison with the current Russian ruler, and he could readily have chosen to start a war when realising that the union was collapsing² – this would have been his most straightforward way of achieving immortality through history.
Queen Christina doing her thing: I’ve always been fascinated by Queen Christina of Sweden. Today, she has become something of a queer icon: she was a tomboy, preferring masculine hobbies and wearing men’s clothes, and declared at 22 she’d never marry³⁴. Five years later, the daughter of Gustav II Adolf, the king who died defending Protestantism in the Thirty Years’ War, abdicated, converted to Catholicism and moved to Rome. A queer queen, indeed.
Abolition of slavery: Just appreciate that slavery was abolished, since the abolition of slavery wasn’t inevitable. More broadly, the abolition of slavery represents the first step big moral circle expansion.
Eradication of smallpox: Big triumph – infectious diseases have played a major role in history. Eradicating an infectious disease is notoriously hard, requiring both scientific advances and international coordination; in fact, only two diseases have been completely eradicated to date (smallpox in 1980 and rinderpest in 2011). Smallpox killed approximately 300 million people in the 20th century alone – and now, no more deaths to smallpox.

The poster for the Leipzig ‘89 - Revolution Reloaded exhibition of the Deutsches Historisches Museum hangs on my wall. And in my hallway, there’s a poem which (among other things) lists a number of noteworthy historical events, such as the glasnost. We must not forget history – not just to prevent atrocities from repeating themselves – but also to appreciate the present.

Historians debate whether convivencia was as utopian as it sounds. ↩︎
But he let his world – the USSR – end not with a bang but with a whimper. ↩︎
Christina wrote a large number of maxims during her lifetime. One of them: ‘It takes more courage to marry than to go to war’. ↩︎
She was learned too, mastering at least five foreign languages in addition to her native Swedish and German. The queen was also keen on philosophy, having Descartes come to Stockholm and give her private lessons at 5am in the winter (the Frenchman soon caught a cold and died ten days later, no joke). ↩︎

On co-living, fractals and life circles

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 04 Jan 2026 00:00:00 +0000

Now that I’m nearing the end of my studies, I’ve reflected a lot on the value of having a tight-knit community, as you always hear recent graduates say they miss the sense of community from college.

I’m mainly interested in communities sharing most aspects of life, ranging from professional to personal – the urban analogue of a kibbutz; I’ll call such communities ’life circles’. Life circles involve standing social events, often at a third space, and its members share a sense of community.

Life circles arise naturally within some Eastern cultures, where they’re usually centered around the family unit, though you’ll also find life circles among creatives, entrepreneurs and other people ‘with character’ in the West. Examples of life circles in the West include Arcadia, Emerge Lakefront and Feytopia. However, a circle doesn’t need to entail co-living: Fractal is just a housing network.

Having a life circle gives you a professional edge. Political, artistic and scientific movements are collective endeavours, and they cannot be carried out by a single person, however smart. Famous examples of life circles include the Bloomsbury group and the Vienna Circle. Similarly, you cannot successfully run a company without having some very good friends – not just coffee-machine acquaintances, but friends you can rely on no matter what. There’s a real sense, then, in which world history wasn’t shaped by individuals, but by groups of individuals.

But above all, feeling a sense of belonging is important for overall life satisfaction, and a life circle provides just that.

Of course – you don’t need a life circle to live well: you can have different friendship groups for the various areas of your life. This creates more overhead, though, and requires you to be both agentic and good with people (I’m neither).

Though Stockholm has among the highest proportions of one-person households in the world (a staggering 60%)¹, there are several life circle initiatives here. The startup ecosystem finds itself in a chicken-egg relationship with life circles, and there are various innovation hubs with life circle vibes: A House, Knackeriet, Alma, Norrsken, Slottsbacken. There are also co-living communities like Emerge Lakefront, K9 and Udda (more examples here).

More broadly, it’s worth mentioning the student apartments and co-living spaces of Stockholms studentbostäder (SSSB), which provide natural meeting hubs for students and young professionals. There’s also a long-standing tradition of study circles in Sweden, some of which also have a life circle ring to them.

Soon-to-be graduates find themselves somewhere between communities based around student apartments and exclusive co-working spaces. If you don’t want to live in a shared apartment, you’re not part of a life circle by default. Or even if you have access to an exclusive co-working space, perhaps you wish to be part of a more diverse community. And there’s no obvious life circle in Stockholm for such people. Problem.

Partial solution (feel free to submit your own solutions here): as a very minimal life circle implementation, you could create a Luma calendar with friends with both standing events and casual one-offs – see e.g. the calendar of Fractal Geneva. I imagine people adding their own events (casual dinners, vibe-coding afternoons, museum visits), as well as events they plan on attending (seminars, plays, concerts). Subscribers of the Luma calendar will then be notified when someone adds an event and can mark whether they’ll join. In addition to the calendar, you may want group chat or a Slack for casual meet-ups.

While you could technically do all event managing through a group chat, group chats don’t scale as well, and they don’t allow for the automatic creation of recurring events. Recurring events form an integral part of any kind of life circle: I wouldn’t count on people organising casual gatherings to keep the circle together. Oh, and you’d also be missing out on the beautiful Luma interface.

I just created a calendar; let me know if I should add you.

This post was inspired by conversations with Agatha Duzan, Elias Koschier, Chiara Gerosa, Konrad Seifert and Åke Lindblom.

This doesn’t translate into higher rates of loneliness, though. In one study, self-reported rates of loneliness among older adults were the lowest in Denmark, Switzerland and Sweden, and the highest in Greece, Israel and Italy. ↩︎

On logging and living

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 28 Dec 2025 00:00:00 +0000

During our family gatherings, my mother loves taking pictures on her iPhone; my father rarely takes pictures unless prompted by my mother (we need photos of her too). I find myself somewhere between: I make an effort to take pictures when seeing family and friends, though I dislike pulling up the camera.

My parents represent two distinct camps: the log and live camps. The loggers are the people taking pictures all the time to record their lives; the livers think that picture-taking ruins the experience and take no pictures at all.

Loggers have been around forever: keeping a diary, making realistic paintings and collecting objects are all forms of logging. Famous examples of logs include the diaries of Samuel Pepys (17th century) and Anne Frank (1944-1942). Serious photo loggers have only been around since the 1950s, when the instant camera arrived.

However, the contrast between the logging and living mentalities is nowhere as stark as within photography, especially today: anyone with a smartphone can easily become a photo logger, so refusing to take pictures is more of a statement. While there have always been loggers, we can’t really speak of anti-loggers outside the context of photography, which is why I’ll focus on photography for this post.

There are many kinds of loggers. The forefather of the logger is the Japanese tourist on the hop-on-hop-off bus with one Nikon on each hip. Now that anyone can take high-quality pictures with their phone, there are more loggers than ever.

Loggers say they take pictures to remember precious moments with families and friends. Since pictures can be made more objective than e.g. diary entries or paintings they offer an easier way to share experiences. However, this doesn’t seem like the only motivation for logging – this doesn’t explain why everyone needs to take their own picture of the turkey at the Christmas dinner.

Loggers often reach for the camera when bored: if you’re fully attentive, you won’t think of taking pictures. Sometimes logging is rooted in a discomfort with doing nothing. Susan Sontag’s remark about tourists also holds for loggers:

Using a camera appeases the anxiety which the work‑driven feel about not working when they are on vacation and supposed to be having fun. They have something to do that is like a friendly imitation of work: they can take pictures.

Logging is also a way to appropriate an unfamiliar situation. When taking a picture, the photographer decides on what to include and exclude, and this feels much like conquering something unknown.

In fact, the photographer exerts tremendous power over the social situation: they can make everyone pause with what they’re doing, move their chairs and put on an awkward smile, the whole affair taking up to several minutes. During the act of photography, the photographer is the centre of attention.

Livers, often the subjects of the loggers, are best understood as anti-loggers.

The livers say that excessive picture-taking ruins the experience, both for the photographer and the subject. Taking good pictures is a hassle: you need to pull up your camera, compose the image, tell everyone to look natural, and redo steps two and three until satisfaction. The subjects grow tired of posing, especially if there are several loggers around – as there often are, since everyone needs their own picture.

More worryingly, however, is how loggers disrupt the social dynamics. If logging becomes the dominating social activity, then a moment is valuable only insofar as it leads to a good photo – picture-taking creates a strong Hawthorne effect. In logging, the logger might also signal that they’re bored, whether it’s true or not: the subjects cannot tell whether you’re scrolling social media or checking whether your cousin blinked. Either way, you’re distancing yourself from the others.

In its purest form, I view logging as immortalising experience, making joyful moments accessible to future selves and future generations. The fundamental tension between loggers and livers has to do with delaying pleasure: loggers prefer savouring a joyful experience for longer, while livers swallow it in one go – it’s like the difference between having americano and caffè normale.

However, I agree with livers that picture-taking should be made less invasive. Loggers can share photos among themselves, rather than having subjects pose for three different cameras; they can agree on norms for when picture-taking is allowed. When seeing friends, I only do portraits just as we’re about to say goodbye – this is my way of indicating that our meeting wasn’t about producing pictures.

A viewfinder introduces a distance. And the logger cannot close the distance by passing their phone or camera around the table and showing the pictures: this distracts from the other social activity. Everyone wants to look at the photos, for photos are attractors of attention. The photographer, then, needs to wield their power carefully.

The varieties of optimal experience

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 21 Dec 2025 00:00:00 +0000

I often wonder what it must have been like experiencing the fall of the Berlin wall first-hand. The photographs of ordinary citizens on the wall flinging their hands into the air give me goosebumps; I bet many eye-witnesses would cite the fall of the Berlin wall as the happiest moment of their lives.

How about the rest of us? On their deathbed, what would the average person view as the happiest moment of their life? Here I use the word ‘happy’ to mean ‘joy’, rather than a sense of ataraxia, so I’m asking about moments of ecstasy.

Although this is personality-dependent, surely there are some patterns to people’s peak experiences. Here’s an attempt at classifying them.

One class of peak experiences have to do with achievements. The fall of the Berlin wall was a remarkable achievement; it signified a victory of freedom over totalitarianism – or even the victory of freedom over totalitarianism, for believers in the end of history. Or consider scoring the decisive penalty during the penalty shoot out in a world cup finals.

Heureka moments also fall into the category of achievements. Einstein famously described the following realisation, at the heart of his theory of general relativity, as the happiest thought of his life:

The gravitational field has only a relative existence… Because for an observer freely falling from the roof of a house - at least in his immediate surroundings - there exists no gravitational field.

But unless you’ve dedicated your life to a professional or religions cause, like Einstein, you’re unlikely to mention an achievement as the happiest moment of your life; instead, most people dedicate their lives to relationships¹.

Moments of nearness with loved ones is the most obvious source of happy moments. Typical examples include becoming a parent, getting married or exchanging ‘I love you’s for the first time.

For moments of nearness, there’s usually an element of novelty or rareness. However, you can also experience extraordinarily joyful moments while carrying out an activity for the millionth time, if you’re in a state of flow.

While there are many kinds of flow, we can crudely speak of either mental or physical flow. Mental flow states include painting, reading a good book or having good conversations; in contrast, physical flow states make crucial use of the body, a good example being runner’s high. Cycling or skiing can also put you in a state of physical flow; as can you-know-what².

Finally, the elephant in the room: drugs. Based on the numerous descriptions of LSD trips in books, movies and music, I assume a few people would mention their best trip as the happiest moment in their life; however, I suspect these people would be in the minority of all people ever having tried drugs. Given that two thirds of Europeans don’t try drugs in their lifetime anyway, I wouldn’t count trips as a separate category.

By now, our taxonomy of highs seems relatively complete. Many people would probably mention a personal milestone, a moment with loved ones or a session of flow as the happiest moment in their life, although the details may vary. One Redditer recalled ‘sneaking away from the group at a church camping trip with your best friend and breaking into an abandoned slaughterhouse while pretending you’re ninjas’ as one of his happiest moments – a very specific instance of nearness and physical flow.

Taking stock of my own happiest moments, two-three moments stand out, and they fall within several of the above categories. But of course, this list is subject to modification – I don’t believe in the end of history.

See Richard M. Berlin’s spin off of Einstein’s realisation, Einstein’s Happiest Moment. ↩︎
Yes. Mihaly Csikszentmihalyi even dedicates a subchapter of Flow to this topic. ↩︎

LLMs and the English Language

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 14 Dec 2025 00:00:00 +0000

Many LLMs have a ’thinking mode’, also called ‘reasoning mode’. Activating thinking mode changes how the model structures its generation: it generates and evaluates more intermediate outputs before its final answer. So, in exchange for more tokens and longer response times, you get a more ’thoughtful’ answer. Thinking mode works phenomenally well, so the temptation to use LLMs for thinking can sometimes be irresistible. However, if I want a genuinely thoughtful answer – if I’m writing for mental clarity – I’ll hardly use LLMs at all.

LLMs can ’think’… #

Here I’ll focus on critical thinking, which, in its broadest sense, is defined as careful, goal-directed thinking¹. This involves things like analysing arguments, identifying relevant information, questioning well-perceived wisdom, and, finally, drawing sound conclusions.

To me it seems like critical thinking skills fall into two categories, which I’ll call ‘wide’ and ‘deep’ thinking. Wide thinking is open-ended and explorative, and aims to gather information; deep thinking is detail-oriented and precise, and churns information into conclusions. Deep thinking is the extraction and evaluation of premises, almost like symbol-free mathematics.

LLMs are very useful for wide thinking tasks. Having been trained on the entire Internet, they know mind-blowing amounts of information, making them invaluable for literature reviews. When self-studying a new topic, I usually begin by consulting an LLM to get a high-level overview – this is the efficient variant of reading the Wikipedia introduction.

And these days, LLMs are also pretty useful for deep thinking. State-of-the-art LLMs already achieve 90% accuracy on benchmarks like GPQA (PhD-level science questions) and MMLU-Pro (problem-solving and knowledge across different subjects). They’re also terrific problem solvers: GPT-5.2 solved all tasks at this year’s AIME, while a version of Gemini achieved gold-medal standards at this year’s IMO².

In short, LLMs can think both widely and deeply, outputting answers reflecting strong critical thinking skills.

…and so can you #

Nevertheless, when I’m writing to understand, rather than to produce an end product for someone else, I barely use LLMs³. I consult LLMs for literature reviews and gladly read the AI generated overviews of search results on Google, but the rest I do myself⁴.

Indeed, the other obvious use cases of LLMs for writing involve outsourcing thinking: I’m responsible for creating outlines, drafting paragraphs and formulating individual sentences – yes, even formulating individual sentences, or else I’m not saying exactly what I mean to say.

This is much like doing the routine checks and exercises for the reader when reading a piece of mathematical writing. To get a gears-level understanding, as is often required, you should work out all the details on a separate piece of paper. Of course, this is rather time-consuming, so I’ll readily use LLMs for routine writing tasks, like emails to colleagues and motivation letters.

In Politics and the English Language (1946), his attack on vague political writing, Orwell argues that the use of ready-made phrases leads to imprecise thinking:

You can shirk it [the pain of thinking for yourself] by simply throwing your mind open and letting the ready-made phrases come crowding in. They will construct your sentences for you – even think your thoughts for you, to a certain extent – and at need they will perform the important service of partially concealing your meaning even from yourself.

I suspect he’s right. And today LLMs can give you ready-made essays and research papers. There have recently been many papers on deceitful behaviour in LLMs⁵, but we can be deceived in many ways.

At least according to the Stanford Encyclopedia of Philosophy. There’s a myriad of more specific definitions, though. This article lists 14 philosophical definitions and three dictionary definitions. ↩︎
It’s still faster for me to typeset proofs myself: if I know the argument inside out, I might as well knock it out myself. However, if I had more money to burn on advanced models and spent an afternoon perfecting a set of prompt templates, using LLMs as TeX slaves would likely save me time. ↩︎
And obviously no LLMs for personal writing (essays, emails to friends, etc.). ↩︎
This is a matter of what I call ‘LLM hygiene’. ↩︎
See here, here and here. ↩︎

Reasons for optimism in technical AI safety

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 07 Dec 2025 00:00:00 +0000

I’ve been doing some technical AI safety research this fall. Technical AI safety is about ensuring AI systems do what they were intended to do, and falls under the broader project of making AI ‘go well’.

If you do technical AI safety research, you spend all day imagining safety failures, and, what is more, you spend all day around AI safety-pilled colleagues. This almost certainly affects your judgement, in some way or other – and this is scary. In particular, it’s easy slipping into hedgehog thinking¹, becoming overly pessimistic and fixating on details. Perhaps one could even speak of ‘AI safety biases’ to denote any such distortions in one’s judgement.

However, when I reflect on the state of technical AI safety, rather than just reacting to information in my day-to-day, I find myself thinking that things are going pretty well. Here are a few reasons for optimism with respect to challenges in technical AI safety.

Safety is a priority #

First of all, the three leading AI labs, OpenAI, Anthropic and Google DeepMind, are all taking AI safety seriously: Anthropic and Google DeepMind both have large teams working exclusively on safety research, while OpenAI is open to collaborating with external parties on safety research². Though there’s room for improvement, these labs acknowledge that safety should be a priority – and this shouldn’t be taken for granted.

There are strong financial incentives to roll out new models as quickly as possible, and yet, state-of-the-art models often undergo long periods of safety testing. For example, OpenAI spent half a year aligning GPT-4, arguably the one model which gave us the biggest leap in capabilities up to date, before its public release in March 2023.

It’s also worth meditating on the rapid change in public discourse about AI risk. Just a few years ago, discussions about AI safety were largely confined to rationalist and effective alturism forums online; nowadays, AI lab CEOs are happy to speak of existential risks from power-seeking AI systems. It’s striking how quickly the views of doomers became mainstream³.

Safety is profitable #

Another reason for optimism is that commercial interests often push towards safer models.

In practise, the dichotomy between capabilities and safety research is often false – a deceptive, scheming sycophant wouldn’t make for a good product. In one of my current projects, which focuses on safety failures in graphical user interface agents, we’ve been struggling to come up with tasks where a highly capable, ’evil’ AI would output something different from a dumb AI model misunderstanding user needs⁴.

The apparent tension between capabilities and safety research can be traced back to the orthogonality thesis, which posits that a superintelligence could pursue malevolent goals – intelligence is ‘orthogonal’ to having good character, as it were. While I think this is a reasonable assumption, intelligence is rarely orthogonal to commercial interests; in reality, they’re often strongly correlated.

Safety is hot #

Finally, technical AI safety has become extremely popular over the last few years, as highligted in Will MacAskill’s article on effective altruism in the age of AI. Jobs in technical AI safety are well-paid, high-status and can offer a strong sense of purpose. Moreover, the day-to-day looks much like the job of an ML engineer, so technical AI safety work naturally appeals to number- and computerphiles. Today, the field isn’t bottlenecked by money or talent, but rather by mentorship – and this is the dream scenario. This is precisely what a booming field should look like.

Adjusting for the AI safety bias #

In summary, I’m quite happy with the state of technical AI safety. It’s growing extremely fast, and I know many thoughtful, highly skilled people, both from industry and academia, looking to pivot to the field. If you have long AI timelines like me, you have good reason to believe that AI safety research will keep existing models in check, so we’ll succeed in building powerful agents doing what they’re intended to do. In brief, I think AI is likely to ‘go well’ – at least on the technical front⁵.

In his 1953 essay, Isaia Berlin introduced the analogy of the hedgehog and the fox to represent two distinct modes of thinking: the hedgehog reasons from first principles and tries reducing complex problems to their core, while the fox acknowledges complexity and tries to aggregate different views. Hedgehog thinking is more effective for solving technical tasks. ↩︎
A good example is the anti-scheming paper from OpenAI and Apollo. ↩︎
See Clara Collier’s excellent book review of If Anyone Builds It, Everyone Dies. ↩︎
For those with some background in AI safety, have a look at the side tasks from the SHADE-Arena paper, which tests models’ abilities to carry out ‘malicious’ side tasks in complex, long-horizon settings. A weaker AI might well execute tasks similar to the malicious side tasks by accident. ↩︎
And my ’likely’ means something like a ≥60% probability. ↩︎

On shame

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 30 Nov 2025 00:00:00 +0000

There are many negative emotions: envy, anger, fear, worry, sadness, to name a few. Positive emotions don’t carry as much nuance. When you’re asked how you’re doing, assuming you’re doing good, you’ll probably reply you’re simply ‘doing good’. Positive emotions like happiness, fulfilment or excitement appear to be more strongly correlated with one another – for this reason, they’re often lumped together under the generic term ‘happiness’¹.

Among negative experiences, worry is the most prevalent. In a 2024 Gallup study where adults were asked which feelings they’d experienced ‘a lot’ the previous day, 39% and 37% reported feelings of worry and stress, respectively. Next followed physical pain (32%), sadness (26%) and anger (22%)².

While moderate amounts of worry might be productive, some unpleasant feelings seem wholly negative. And the worst of unpleasant feelings is shame.

Shame #

Shame is unlike worry, stress, pain, sadness or anger; it’s sometimes defined as an uncomfortable feeling of guilt, and is often related to a loss of respect. For me, the most common source of shame is mistakes. Most such mistakes are inconsequential – even the worst of typos won’t kill – but they impinge on my honour. I feel particularly ashamed when reflecting on social blunders, times when I failed to respect norms or hierarchies.

Coping with shame #

Part of why shame is so insidious is that you can’t do much about it: when you’re ashamed, part of you is playing the role of the admonishing teacher. You cannot make yourself shameless, as it were, because that would involve contradicting yourself. Sometimes you can get rid of shame by redoing things the right way, though this is impossible for many social situations. So, you err, you’re ashamed, you learn, and you’re still ashamed.

This said, sometimes you shouldn’t learn: you shouldn’t necessarily let shame influence your behaviour. There are many wicked social norms and beauty ideals causing shame, and these must be ignored. For example, I sometimes feel ashamed for asking ’too many’ questions during lectures, even when they’re well motivated: women shouldn’t take up too much space, and especially not in the context of mathematics. Another example: I sometimes have to remind myself I’m free to not shave my legs. A complicating aspect of shame, then, is that you can be ashamed although you’re right and they wrong.

The intensity of shame #

The worst thing about shame, however, is its intensity. In my experience, no other negative emotion can be as violent. Even moderate amounts of shame can be unbearable, making you want to vanish from the surface of the earth for a while. The force of emotion can being debilitating, preventing you from learning whatever lessons are to be learned.

And whereas intense anger or sadness can be pleasurable, a conundrum first discussed in Poetics, shame just feels terrible. There are no tragedies, sad movies or sub-genres of emo music meant to elicit shame – shame doesn’t lead to catharsis; shame isn’t pure emotion. Instead, shame responses are complicated and highly culturally dependent.

Societies of shame #

Shame is a powerful driver of human behaviour, and it has been exploited to enforce social institutions since forever. For instance, shame compels people to be faithful to their spouses and respect social hierarchies, and most successful world religions have incorporated an element of shame to ensure their growth. Societies are largely built on shame, especially those where there’s a strong culture of honour.

So shame serves an important role, but only on a social level. Most other negative emotions can benefit the individual; shame only benefits the individual insofar as it increases their social status.

Humans are sometimes willing to inspire anger, fear, envy – any negative emotion really, except for shame. As noted by Hjalmar Söderberg in Doctor Glas, we’re desperate for human contact:

One wants to be loved; failing that, admired; failing that, feared; failing that, hated and despised. One wants to instill some sort of feeling in people. The soul shudders at the void and wants contact at any price.

And shame is the opposite of human contact.

I’m just re-expressing the opening sentence of Anna Karenina in a roundabout way: ‘All happy families are alike; each unhappy family is unhappy in its own way.’ ↩︎
Compared with the 2006 data, all numbers have risen, on average by 7 percentage points, though the ranking of negative feelings has remained pretty stable. ↩︎

Normal situations, abnormal norms

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 23 Nov 2025 00:00:00 +0000

Most people lead a comfortable 9-5 life. You go to work, work away, go home, sometimes seeing friends or engaging in extracurriculars. Even so, sometimes you find yourself in unusual social situations, situations where a first-timer wouldn’t know how to behave. Here are some normal situations with abnormal social norms.

Flights: On a 12-hour flight, you find a cross section of the population bored, tired and without Internet. In this willpower-reduced state, snobs dress in sweatpants and workaholics watch thrashy feel-good movies. A surprising number of people try napping¹, while others strike up conversations with strangers.
Dates: It’s common for people taking steps to get to know new people, for example by attending local meet-ups; however, friend-seekers rarely try getting to know others well. Dates are among the few situations where people deliberately try building deep connections. Think about it: the purpose of a date is for two people, usually strangers, to become romantic partners. Pretty remarkable, right? Even more remarkable is that it sometimes works.
Alcohol: Alcohol creates a form of social anarchy. Moderately tipsy people are amusing – they’re a bit like children, candid and ignorant of social norms, though they enjoy discussing grown-up topics.
Work interviews: A work interview is nothing at all like an ordinary conversation, despite appearances; it’s like an improv play where the interviewee tries selling themself and the interviewer maintains a poker face. The interviewee presents themself as the ideal employee: competent, reliable and a good coffee machine conversationalist².
Music festivals: During music festivals, thousands of sweaty people packed like sardines scream in unison while jumping up and down. Though they’re strangers, they share a sense of camaraderie, sometimes even sharing food and belongings³. Cult?
Summer camps: There are summer camps for kids as well as adults, though the latter are sometimes called ‘retreats’ or ‘get-aways’. The goal of most summer camps is for a group of strangers to pursue a common interest together and make new friends. Being in a remote place helps create a space for intimacy: the camp participants tacitly agree that ‘what gets said here stays here’.
Military service: Life in the military is famously totalitarian: it’s all about obeying your superiors. And if you do mandatory military service, you’re normally at the bottom of the pecking order. In no other setting is there such a clear hierarchy, and in no other setting is outright bullying socially acceptable.

These are all time-bound activities disconnected from everyday life, usually physically or by means of clothing, with extraordinary social norms. This makes them good settings for people-watching. Why study ‘man in a state of nature’ – an impossibility anyway – if you can study ‘man in a state of society’?

Perhaps this is telling of our society. ↩︎
Any weaknesses of the interviewee are, naturally, just their virtues in excess: ‘I’m somewhat stubborn, but that’s just because I’m so driven.’ ↩︎
Every music festival has a bit of Woodstock to it. ↩︎

Techno-optimism - techno-altruism?

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 16 Nov 2025 00:00:00 +0000

You might think the average worker in the tech industry is a diehard capitalist, since tech is full of rich entrepreneurs. Moreover, many big tech companies appear to be driven more by money-making than by providing consumer value: Facebook didn’t connect us with friends and family and OpenAI didn’t remain a safety-oriented non-profit.

However, most tech hubs are overwhelmingly liberal: 64% of voters in San Fransisco are registered as democrats; 8% as republican. Many programmers also dream to create widely accessible software, software for anyone, rich or poor, with stable Internet connection.

Sometimes I think that the grandfathers of communism would be tech bros had they lived today. Technology has allowed for the scaling of selfless, altruistically motivated side projects. Here are three notable examples.

Open source software #

Open source software, code freely available for anyone to use or modify, is a remarkable phenomenon. Although researchers routinely shared code since the early days of computing in the 1950s, open source became a thing in the 1990s. Today open source software is used in many critical technologies; for example, Python, Git and OpenSSH (the most popular SSH implementation) are all open source.

One 2024 Harvard study estimated the overall value of open source to be $8.8 trillion, about the GDP of France, Germany and the Netherlands combined. The study also estimated that firms would have to spend 3.5 times as much time on software development if open source wouldn’t have existed. Though you should expect huge error bars to these numbers, it’s clear that open source generates a lot of value.

We’re lucky things panned out this way – the success of open source doesn’t seem inevitable. Considering the value they generate, open source developers receive way too little credit. They’ll have highly starred repos on GitHub, which is kind of a flex, but that’s about it; meanwhile, their peers earn six-figure salaries. Also, remember that iOS and Microsoft are closed source, as well as most of the cutting-edge AI models¹. So, one could certainly imagine all the buy-me-coffee buttons being replaced by paywalls.

OpenCourseWare #

In 1999, the University of Tübingen in Germany published a series of lectures freely available online, marking the beginning of the OpenCourseWare (OCW) movement. Three years later, MIT launched their OCW, the most ambitious OCW up to date, now offering 2.500 courses with lecture notes, problem sets and exams. Other American elite universities soon followed, publishing learning materials from their most popular courses online. There are also joint OCW initiatives, like edX, allowing anyone to audit courses on topics ranging from corporate finance to aerospace engineering.

And so we find ourselves living in a world where the average person can get to the level of a first-year undergrad in any subject without leaving their bedroom². Considering the state of human knowledge in 2025, having done one year of bachelor’s in three to five subjects, you’d know more than most Renaissance men.

Wikipedia #

Wikipedia was launched in January 2001 by Internet enthusiasts Jimmy Wales and Larry Sanger. There had been previous attempts at creating free online encyclopaedias, starting in 1993 with Rick Gates’ initiative to create Interpedia, or The Internet Encyclopedia. In March 2000, Wales and Sanger founded Nupedia, an encyclopaedia written by volunteers with subject-matter expertise. Nupedia didn’t take off, largely in part because of the extensive peer-review process, and the idea behind Wikipedia was for people to submit entries that would go into Nupedia’s peer-review process. But Wikipedia exploded: while Nupedia had only received 12 articles when Wikipedia was launched, Wikipedia had 18.000 articles in January 2002.

Today Wikipedia is absolutely huge. As of November 2025, the English Wikipedia has around 7 million articles. By contrast, Britannica has fewer than 150.000 articles³. Also, considering that anyone can edit Wikipedia, the articles are surprisingly high-quality: a 2005 study found that the average science article in Wikipedia contained four inaccuracies; Britannica, around three.

It’s worth meditating on this for a moment. Who could have predicted the success of Wikipedia? Imagine the amount of expert human labour that goes into each article – labour which, just as for open source development, goes unrecognised. Today you can learn anything about everything by googling, and this is largely thanks to Wikipedia.

Final words #

Open source, OpenCourseWare and Wikipedia all rely on the Internet, and in turn, on brogrammers happy to spend their weekends doing unpaid work⁴. These projects fundamentally depend on people’s goodwill.

It’s wonderful seeing open source facilitating collaborative development and OCW and Wikipedia enabling lifelong learning. However, what I find most fascinating about these projects is that they demonstrate that pure altruism is possible at a large scale. Considering this, I can imagine Marx smiling in his grave.

More generally, considering that the drug discovery process seems so closed, you might marvel at the mere existence of open source. ↩︎
In-person learning is far more effective than online learning, mostly because of the social context, so I wouldn’t go so far as saying that you could learn the content of a full bachelor’s remotely. ↩︎
On Britannica’s website, it says that members gain access to “over 130.000” articles, so I assume the number of articles is below 150.000. ↩︎
Perhaps the most striking feature of these projects is that they all began as side-projects. See how productive slack can be? ↩︎

What effective altruism misses

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 09 Nov 2025 00:00:00 +0000

Effective altruism (EA) centres around a dozen or so “cause areas”, areas in which you can have a big positive impact. Common cause areas include AI safety, global poverty, farm animal welfare, biorisk, nuclear security and EA field building. As of a couple of years, AI safety is very hot within EA, with some of the EA golden boys, like Will MacAskill and Holden Karnofsky, making AI going well their top priority.

Say that EAs solve the problems associated with the above cause areas. Humanity has smoothly transitioned to a post-AGI world, eliminated global poverty, put an end to factory farming and reduced existential risk to zero. While this sounds like a rosy vision of the future, unless you live in extreme poverty, your life in this hypothetical world will be the same. Those who regularly worry about all the world’s problems will find something else to worry about. And if you are dissatisfied with life – which, if you’re anything like the average American, isn’t entirely unlikely¹ – you’ll likely remain dissatisfied.

EA, then, appears to miss something very fundamental about what it means to be human². If EAs want to maximise positive impact, they should make wellbeing a core cause area.

What I mean #

The cause area of wellbeing, as I see it, could be concerned with the following question: How can we build better societies, where the average citizen has a higher baseline level of wellbeing? Here, I use the term “wellbeing” to mean “mental health” or “welfare”.

Imagine a society organised around the principle of improving wellbeing in the population at large, rather than around some ideology. While I don’t know exactly what such a society would look like, current societies leave a lot to be desired.

Some ideas that come to mind. Can we create societies where everyone feels a sense of community? Societies where people report high levels of life satisfaction well into their 80s? Societies where teenagers take care of one another?

Designing such societies is a long-term goal. In the near-term, we must fix broken healthcare systems, e.g. by providing widespread access to therapy. In America, about 50% of adults with mental illness receive no treatment³.

Why EAs should care #

EAs have good grounds to make wellbeing a core cause area. An SNT analysis reveals big potential for positive impact.

To begin with, the “wellbeing problem” is huge in scope. According to one estimate, mental health disorders accounted for about 16% of global DALYs⁴ in 2019. If everyone had a baseline level of mental health, we’d see many positive downstream effects. Of course, EAs have estimated the benefits of merely treating EAs suffering from depression, making them more effective in their altruistic endeavours. Moreover, as material living standards increase, mental health problems might become the biggest obstacle towards humanity’s flourishing. Who cares if we can colonise other galaxies if one in six men reports being lonely?

The topic of mental health also seems neglected in EA circles. The classic cause areas, along with AI safety, dominate the EA discourse. To my knowledge, there are only two wellbeing-oriented EA organisations: Rethink wellbeing, which offers therapy to altruists, and the Happier Lives Institute, which finds cost-effective charities to improve wellbeing. However, I’d love to see EA think tanks focused on designing happier societies. Designing happier societies is a huge social engineering task, where an EA mindset of truth-seeking and scope-sensitivity would be useful.

Finally, improving wellbeing appears to be a relatively tractable problem. Sound decision-making in healthcare can have seismic effects, and now, with a growing body of research on wellbeing, policy-makers can make more well informed decisions. Efficient policy advocacy might be hard, but it shouldn’t be any harder in healthcare than in AI⁵.

Towards a better future #

If EAs want to maximise happiness, they should cut to the chase, making wellbeing a central cause area. Once those who need help receive it, think about how to design better societies. Don’t aim for utopia – for a start, make teenagers happy again, eliminate male loneliness, and restore the elderly’s sense of dignity. While everyone experiences – and probably should experience – some degree of existential angst, a society doesn’t need to have depressed incels, burned-out careerists and disoriented NEETs.

I agree with the sentiment that EA is becoming somewhat outdated, expressed by Will MacAskill in his article on EA in the age of AGI. In order for EA to stay relevant, the movement needs to reprioritise cause areas. However, rather than morphing into a subfield of AI safety, EAs should ask what would fundamentally create a better world.

One in five Americans say they’re dissatisfied with life. ↩︎
Maybe the best evidence for this claim is that EA only attracts certain kinds of people. ↩︎
In practise, therapy is restricted to those with strong negotiation skills, who can convince healthcare workers they need priority, and CEOs, who receive therapy in the form of CEO coaching. ↩︎
DALYs, disability-adjusted life years, roughly measure the overall negative effects of a disease. ↩︎
I have limited experience with policymaking though, so take the point about tractability with a grain of salt. ↩︎

The Sanremese

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 02 Nov 2025 00:00:00 +0000

In Europe, you don’t have to travel far to experience a completely different culture. If you randomly select a point in Europe outside of Russia, you’re never further than 10h by train from the nearest nation border. There are 44 countries and 24 official languages in Europe – yet, the area of Europe is just 60% of the area of the American South.

I expected Sanremo to be like some of the towns by the French Riviera I’d visited when living in Aix, only with better ravioli and friendlier people. Although I wasn’t entirely wrong, I was far from correct.

Founded in Roman times, Sanremo became a resort destination in the mid-1800s, hosting European nobility from Russia and Prussia. These days, tourism remains the primary source of economic activity, and, just like the average tourist destination by the Mediterranean, its city center is dominated by restaurants and clothing boutiques.

However, Sanremo has a few peculiarities distinguishing it from some of the similarly sized towns on the other side of the French-Italian border. For example, much of the city smells of flowers, earning it the nickname “La Città dei Fiori” (“The City of Flowers”). But the biggest differences come from observing people on the streets, the Sanremese.

The first thing you’ll notice is that the posh Sanremese wear Gucci rather than Louis Vuitton. The ladies don’t try being chic, as in some of the coastal cities in France invaded by Parisians; instead, they dress in Italian designer clothes. Some of them seem to dress to maximise the number of logos visible to passersby. Among men, beige suits aren’t as popular as in Nice. Men in Sanremo make more efforts to be macho, often wearing tracksuits or tight-fitting Armani T-shirts.

However, the rich in Sanremo don’t own the city, as they do in Monaco or Nice. Most people on the streets are perfectly normal Italians. In fact, I met some stereotypically Italian Italians: I met grannies 120 cm tall, grandpas with flat caps and pipes and men with giant calves on Bianchis.

Not only are the tourists in Sanremo fewer in number, but they are also of a different kind. On the French Riviera, most tourists come from either Britain, Germany or America. During my three-day stay, I heard almost no English or German. I heard a lot of French though – this is hardly surprising, given that everything is 20% cheaper on the Italian side of the border.

In Nice and Sanremo, the proportion of foreign residents is somewhere between 10 and 20%. While foreign residents in Nice mostly come from the former French colonies, Sanremo has a large population of Eastern Europeans. In the late 1800s, there was a particularly strong Russian community in Sanremo – one of the city’s most famous buildings is the Russian Orthodox cathedral from 1912.

Despite having a population of just 55,000, Sanremo is surprisingly diverse. It’s neither a dead rural village nor a tourist enclave like some of the French towns 50 km West. There’s a good balance between locals and non-locals, between normal and abnormal. Sanremo feels like one of those resort towns from the 1900s that’s actually wholesome.

On cities

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 26 Oct 2025 00:00:00 +0000

The 17th century was a century of rapid human progress. There was the Scientific Revolution. Philosophers like John Locke and Thomas Hobbes laid the groundwork for modern political thought, while René Descartes and Blaise Pascal made important contributions to philosophy more broadly. Meanwhile, in the arts, William Shakespeare redefined English literature, while Rembrandt changed the way artists approach their subjects.

The above mentioned people have something important in common, other than being extremely intelligent white men. They spent non-negligible parts of their lives in cities. In the 1600s, only 8% of the European population lived in cities. Many people spent their entire lives without setting foot in a big city. But there is a sense in which you need to spend some time in a city to become an influential historical figure, or at least if you wish to understand human societies. Cities tell thousands of tales about history, culture and economics.

In history #

To begin with, cities serve as historical documents. A city’s architecture provides insights into the lives of its inhabitants, both of the past and present. Consider, for example, the massive apartment complexes of East Berlin or the colourful houses perched on the hills in Valparaíso.

Monuments tell us about the values, artistic ideals and technologies at a given time in history. The Blue Mosque of Istanbul marks the historical influence of Islam, while the Dohány Street Synagogue in Budapest marks the influence of the Jewish community in Budapest. The Burj Khalifa is a remarkable engineering achievement, as well as a symbol of wealth and vanity.

A city might also contain more explicit historical references, such as memorials. Memorials, being explicit expressions of regret, grief and triumph, reveal the citizens’ relation to history. The Monument to the Discoveries in Lissabon, built as late as in 1960, is a glorification of the country’s colonial past.

In culture #

But cities aren’t just of interest to historians. Movies, books and plays need to take place somewhere where there’s life, and there’s no more life than in a global metropolis. Rural areas are sometimes referred to as “dead”, while New York never sleeps. In fact, many pieces of narrative art appear to be about the city itself, the plot being a mere formality. Some novels are riddled with location references to the point of becoming difficult to follow for non-locals; consider Ulysses, for instance.

Cities often referenced in popular culture become cities of our imagination, making them tourist magnets. Paris owes a large part of its tourism to the innumerable cultural references to the French capital. Indeed, “American expats loafing in Paris” could almost be considered its own literary genre.

Just as the characters they create, artists are also products of cities. This has given rise to another kind of cultural tourism, whereby visitors visit key locations in an artist’s life. Prague doesn’t just brand itself in terms of cheap beer, but also as the city of Franz Kafka.

In economics #

Some cities aren’t defined by their historical and cultural heritage, but rather by economical factors. Such cities shed light on how the global economy works, as well as the human desire to make it big. For this reason, many accounts of economic history involve lengthy descriptions of cities.

To analyse cities from an economical viewpoint, Max Weber introduced the distinction between “producer cities” and “consumer cities” in his 1921 article The City. Producer cities are organised around some specific trade, while consumer cities are rather like administrational centres with residents having access to certain streams of income. Think Detroit versus Washington DC or, for a European example, Antwerpen versus Brussels. An extreme example of a producer city is Kiruna, a city in the north of Sweden, built in the 1890s to serve the Kiruna Mine.

Certain cities are defined by the human capital they attract, the quintessential example being San Francisco. Startup founders pilgrim to Silicon Valley, since that’s where everything happens. For many San Franciscans, The City by the Bay is more of a transit point than a proper home. University towns are similar, in that a large part of its population aren’t permanent residents.

Understanding people through cities #

Much of human innovation, whether it be in culture, economics or arts, takes place in cities. Wonders happen when smart people with diverse backgrounds form tightly knit communities, as in, say, Bloomsbury in the 1910s. Such communities are more likely to arise in cities. However, this isn’t the sole reason cities are important drivers of human progress. To drive real human progress, you need to understand humans and the societies they build. If you want to understand people – their fears, desires and rituals – understand cities.

Cities are storytellers. They tell you thousands of stories in a most engaging way, all at once. Even a moderately sized city can tell you everything you’ve ever wanted to know. You just have to listen.

Thanks to Raphaëlle Cohen, Emil Wasteson Wallén and Miles Kodama for feedback on this text.

The mad scientist is probably mad

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 19 Oct 2025 00:00:00 +0000

Many pioneering scientists were initially rejected by the scientific community. At first, the ideas of Galileo Galilei, Gregor Mendel and Geoffrey Hinton were ridiculed. There are many more examples of such “martyrs of science”, people who were cancelled and then vindicated.

Science isn’t as dramatic nowadays. There aren’t as many Galilei-like figures, at least not in relative terms. If 1/1000 of crazy-seeming scientists turned out to be geniuses in the 20th century, today the proportion might be more like 1/1 000 000. Why is that?

To begin with, it seems as if good ideas are getting harder to find. There are some ideas whose discovery appears almost inevitable because of the amount of readily accessible data. In contrast, much science today requires costly equipment and large-scale collaboration. Scott Alexander put it well in his essay about science slowing down:

For example, element 117 was discovered by an international collaboration who got an unstable isotope of berkelium from the single accelerator in Tennessee capable of synthesizing it, shipped it to a nuclear reactor in Russia where it was attached to a titanium film, brought it to a particle accelerator in a different Russian city where it was bombarded with a custom-made exotic isotope of calcium, sent the resulting data to a global team of theorists, and eventually found a signature indicating that element 117 had existed for a few milliseconds. Meanwhile, the first modern element discovery, that of phosphorous in the 1670s, came from a guy looking at his own piss.

Early scientists picked the low-hanging fruit.

What’s more, the average scientist today is more competent than the average scientist 100 years ago, since job selection has become more meritocratic. In the 18th and 19th centuries, science was largely run by aristocrats with too much spare time. These days, becoming a leading researcher in a STEM field requires brains. So if I’m disagreeing with a great number of brilliant people, it’s more likely that I’m the one being wrong.

Finally, the industrial organisation of science might create a selection pressure against wild ideas. While most wild ideas are bad, brilliant ideas often seem wild too. Across almost all scientific disciplines, researchers publish more papers now than a hundred years ago. Someone optimising purely for research output probably can’t afford to explore as many wild ideas. If you’re expected to publish 4-5 papers per year, you’ll spend a significant amount of time on overhead work, leaving you with less time for open-ended exploration. In terms of Kuhn’s framework for scientific revolutions¹, you’ll mostly be doing normal science.

Science isn’t what it used to be. The fraction of martyrs of science today is minuscule. This doesn’t mean today’s scientists are any less intelligent². Rather, it means that today’s scientific methodology works relatively well, and that brilliant ideas are the norm rather than the exception. If we’re right about the mad scientist just being mad, that’s a good thing.

This post was inspired by a conversation with Miles Kodama.

According to Thomas Kuhn, science goes something like this: Most of the time, scientists do normal science (experiment, analyse, repeat). But with time, they start noticing anomalies, results which cannot be explained by existing theories. Soon, there is a scientific crisis, leading to a scientific revolution. After the revolution, you’re back to normal science again. The people calling out their colleagues for being wrong should be prepared to be cancelled, at least for some time. ↩︎
According to Claude, the number of living physicists outstrips the number of having physicists having lived in the past, from ancient times until the present day. So there are likely many physicists of the same calibre as Einstein, Schrödinger and Bohr alive today. ↩︎

High school for heroes

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 12 Oct 2025 00:00:00 +0000

High school isn’t so much about preparing teenagers for adult life. In Sweden, you have a subject about cooking and personal finance, but only until secondary school. I think this makes sense: you only need one course on practical adult skills. After you know the basics, it’s all about practising IRL.

But being a competent grown-up involves many other skills. It’s not just about successfully running a household – it’s also a matter of “running yourself”. There are plenty of “serious” subjects we could fit in a standard high school curriculum. Here’s a vibe-based list:

Learning: Please please please, teach high schoolers about spaced repetition, interleaving and testing. The ability to learn things quickly is useful in any domain of life. In the future, this skill will probably become even more useful, as AI speeds up social and technological progress.
Rationality: If you want a serious-sounding name for this subject, call it behavioural economics or applied philosophy. Basic knowledge about cognitive biases and System 1 and System 2 isn’t just intellectual amusement – it’s also very practical. If everyone learned about confirmation bias at an early age, would there be as much polarisation in the world?
Curiosity: We could also make high-schoolers read non-fiction or watch videos on popular science. Make them read or watch videos about the French revolution, the opioid crisis in the US, personal effectiveness, the holocaust, the nature of time, the Bauhaus school of design, John Houdini… They could also organise seminars for each other, as in Fractal. Forcing students to memorise facts won’t produce life-long learners.
Statistics: Most people are forced to take a stats class at university, and for good reason. Statistics is one of the most useful academic disciplines, and it forms the backbone of all science. Maybe we could teach high schoolers about Bayesian inference, hypothesis testing and confidence intervals. This material can be made accessible to high schoolers: you can explain Bayes’ rule by means of rectangles and teach students simple algorithms for hypothesis testing and computing confidence intervals.
General AI: I almost feel obligated to add something directly AI-related. I think students should be taught the basics of machine learning: how gradient descent works, general transformer architecture, etc. This is much as students are taught about evolution and the different parts of the cell. I think this would empower people to reason more clearly about transformative AI¹.
Wellbeing: You might also offer a course on wellbeing, incorporating elements of philosophy and the science of happiness.
Agency: We could also teach a class on on agency, by which I mean the art of “doing stuff”. Most high-schoolers have never experienced the joy of agency, the joy of poking at the world and noticing something happening. One could e.g. have a project-based class where high-schoolers get to found their own companies or orgs².

I’m the most bullish about the benefits of taking classes on learning and rationality. I’m not as certain about the benefits of the remaining subjects. So, here’s one suggestion for a high-school curriculum: take a standard high school curriculum, make learning and rationality mandatory and offer the remaining subjects as electives. Overall, this seems like this kind of curriculum would produce more well-rounded, curious doers – the kind of people who might solve the world’s most pressing problems.

You might also consider adding a modules on prompt engineering. ↩︎
Some Swedish high-schools offer such courses in entrepreneurship; Google “Ung företagsamhet”. ↩︎

A study on slack

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 05 Oct 2025 00:00:00 +0000

I. #

Once I was done with exams, it felt like getting my brain back. It’s not just that you spend most of your day studying difficult subjects, leaving you with less time and willpower for other tasks. You also have to do cost-benefit analysis all the time. All. The. Time. Sometimes, it’s relieving if someone tells you exactly what to do¹.

II. #

Slack is sometimes defined as having leftover resources. Intuitively, slack is like a buffer allowing you to absorb shocks better. We often speak of slack in the context of time and money. But to make things more visceral, imagine packing a bag. Having slack means having leftover space. There’s no need to stuff your shoes with underwear, fold your clothes meticulously or pack things in a given order. You don’t have to engage in trade-off thinking (“If I leave my camera, I can fit my notebook and an umbrella”). Having slack leads to a feeling of abundance.

Slack is trendy nowadays. The concepts of “slow productivity”, “intentional work” or “anti-hustle culture” are essentially about having more slack. There has also been some recent research into the benefits of slack.

Some companies have tried implementing four-day work weeks and measured productivity and employee wellbeing. The results? One survey paper found that implementing four-day work weeks also made employees happier and healthier. This also agrees with my personal experience. I can only do good mathematics for at most 4-5 hours per day. After that, I need a long break.

As stressed in Scarcity, having slack also improves judgement. If you have little time or money, you tend to tunnel – neglecting anything which isn’t immediately related to that resource (evolutionary baggage). There have also been numerous experiments showing that scarcity reduces mental bandwidth, i.e. intelligence and impulse control.

Perhaps slack is necessary for being happy too. Chronic stress and financial insecurity have very negative consequences on wellbeing.

III. #

To create slack, just say no.

Saying “no” is hard, though. Planning fallacy often leads us to take on too many tasks. While there are ways to overcome planning fallacy², everything takes longer than expected by default. Even when you add a planning fallacy allowance. Moreover, saying “no” requires prioritising – cognitive effort.

To say “no”, remember that slack is the best thing ever. Here you can tap into your own quirks. For example, I often slip into a productivity-maxxximiser mindset, so I’ll try reminding myself that slack makes me more productive. I also have a few idols, and creating slack also seems consistent with my goal of becoming more like them.

But I don’t just want to rely on mental habits. To create slack, I’ve set myself the goal of “working as an adult” rather than a student. That is, I’ll fix reasonable working hours, take Sundays off and treat breaks as a non-negotiables. These habits are all meant to protect slack. I also think digital hygiene helps create slack by making us more efficient. I’m wary of email and Slack checking, I keep my phone out of reach when studying and I mostly keep my phone on silent.

I’ve also tried identifying early warning signs that I need more slack. For example, when I have little slack, my message respond time increases from my targeted 24h to around three days³. I also tend to cut back on my reading before going to bed and clean my room less frequently. Silly, I know. But these are surprisingly effective indicators of my stress levels.

IV. #

It feels good having one’s brain back. To do good work, you to be working at full capacity. This doesn’t mean overexerting yourself. Rather, it means having slack. Good work – sustained efforts to solve hard problems effectively – comes from the feeling of slack.

This post was partly inspired by conversations with Yannick Muelhauser, Agatha Duzan and Hatem Khrouf.

One of my friends thought this was one of the main benefits of doing military service for a year. ↩︎
A few ideas: ask a friend or Claude for a second opinion or recall how long similar tasks usually take. ↩︎
If I don’t respond to a message within 24h, feel free to message me “You need more slack!” ↩︎

Analytic podcast-listening

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 28 Sep 2025 00:00:00 +0000

Nowadays, you can spend all your waking hours consuming valuable information. Pretty remarkable, when you come to think of it. Hint: AirPods Pro.

Whenever you’re doing something not demanding your full attention, you can always turn on a podcast. You can listen to podcasts when commuting, working out or cooking¹.

Technically, you could have done something similar 50 years ago using a walkman. But it have been very high-effort. Today, we have dedicated podcast apps and noise-cancelling, wireless earplugs. As a result, the podcast industry is rapidly growing in size. If you sample 100 young adults from the street in an urban area, a substantial number of them might be listening to podcasts.

I. #

In school, they teach you analytic reading. But they don’t teach you analytic podcast-listening. By this, I mean podcast-listening to learn about new ideas. Deliberate listening, as opposed to just hearing.

Analytic podcast-listening is very different from analytic reading. While I find it harder following the structure of an argument in podcast form, the podcast medium has other benefits:

Podcasts are low-effort way of discovering new ideas. Because there are so many podcast hours in the day, you can afford to explore more novel content. I’m pickier when selecting books to read, since it’s a bigger investment.
Podcasts can have a very high idea density. Because most podcasts are at most one hour, podcast guests are forced to focus on their key insights.
Podcasts are great for stress testing ideas. Someone might make a convincing argument in a blog post but fail to defend their views in a podcast. Similarly, you might come away from a podcast feeling like a podcast guest is completely unreasonable. Suspicious!

So, it seems worth practising analytic podcast-listening. If you listen to podcasts 2h daily, this might make a huge difference.

II. #

Before we get into strategies for analytic podcast-listening, I’d like to stress the need to prioritise. If you feed a neural network bad data, you won’t get good results – no matter how efficient the training process. My podcast taste, in a nutshell:

I mostly listen to podcasts with transcripts, so I can easily refer back to interesting parts of the conversation. For my favourite episodes, I’ll chuck the transcript into a large language model and have it summarise the key points. Ideally, the show notes also contain time stamps and further references. In this respect, Hear This Idea and the 80,000 hours podcast do an excellent job.
I find it quite valuable listening to podcasts with authors of books or articles I’ve read. These podcast episodes serve as good refreshers. This point also relates to the above point about stress testing ideas.
I also like listening to blog posts in podcast form. While I don’t retain as much information as after reading the post, I usually get the gist. Above all, listening to blog posts helps me decide which posts to read later.

III. #

Here are some strategies for analytic podcast-listening²:

Have some system for saving podcasts that resonated with you. Re-listen to your favourite episodes, much as you’d reread your favourite books. Here’s my podcast shelf.
Similarly, keep an anti-library for podcasts. I have a list of podcasts recommended by others.
Create voice memos to capture your thoughts. If I’m on the go, I might dictate thoughts and have an LLM convert my yapping into bullet points later.
There are dedicated podcast apps which allow you to capture segments of podcasts. I’ve played around with some of these, and I’m quite impressed by how well they work.
Work with playback speed. Here it’s useful to think in terms of exploration and exploitation. If it’s a high-quality episode, I’ll usually listen at a 0.8x speed. While listening to podcasts, I’m usually multitasking, so I need additional processing time. If I’m listening to a new show, I’ll turn up the speed to 2x.
If you have a smartwatch, put it to good use! The most valuable feature of my Apple Watch has been the ability to rewind podcast episodes by 15s.

IV. #

Podcasts play a big role in my process for updating beliefs. Podcasts, along with conversations with others, are my main methods for “gaining surface area”.

I only began thinking in terms of analytic podcast-listening quite recently. Ultimately, I’m realising that it’s “just” a matter of being more intentional about the content you consume. But it probably helps having a funky name for it.

Listening to podcasts at all times is a bad idea, though. ↩︎
If you have further ideas here, please do reach out. ↩︎

AI safety lingo

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 21 Sep 2025 00:00:00 +0000

There’s a lot of jargon within AI safety. Here are analogies for 20 AI safety terms. I assume some familiarity with these terms; I’ll omit the exact definitions. I’ll give references to appropriate resources rather than trying and failing to define these concepts precisely in a couple of lines. Instead, I’ll focus on intuitions.

Embedded agency: Playing the Sims is very different from living IRL. When playing a video game, you’re not an agent embedded in the game environment.
Base optimizer vs. mesa-optimizer: A base optimizer is a process for achieving some goal (cooking a good risotto). The base optimizer (the chef) soon learns that the seasoning makes a huge difference. A process for perfecting the seasoning is an example of a mesa-optimiser: a process for achieving a learned subgoal.
Inner alignment vs. outer alignment: If a government wants to reduce unemployment, it has to design efficient regulations and and ensure citizens comply. Outer alignment is the problem of specifying the right incentive structure; inner alignment the problem of compliance.
Deceptive alignment: This is much like an intelligent bully will pretend being nice when the grown-ups are watching. A misaligned AI system might benefit from appearing more aligned.
Treacherous turn: The moment when the opposition seizes power through a coup. The hypothetical moment when a misaligned, highly capable AI decides to… I don’t know, but people imagine something bad.
Corrigibility: Ever been in the library when someone’s phone continues ringing, despite their best efforts to silence it? They might try putting the phone on silent, then turning the volume to zero, then turning it off. Maybe their phone seems to be frozen or something? In this case, we’d speak of a non-corrigible phone: a phone that resists attempts to “correct” its behaviour and resists attempts to be shut down.
Goal misgeneralization: A pianist might practise Bach to please her pianist friends; however, when she’s at normal parties, people just want her to play Let it Be. Likewise, an AI system might competently pursue one goal which leads to good performance in training situations but poorly in novel test situations.
Edge instantiation: An AI agent instructed to fill a cauldron of water might flood the entire room. Task accomplished, technically. And for AI agents, only the technicalities matter. AIs can be annoyingly creative.
Goodhart’s law: When a measure becomes a target, it ceases to be a good measure¹. The phenomenon of studying for the test is an example of Goodhart’s law.
Value learning: This is the broad project of teaching AIs human values. AI-rearing, essentially.
Inverse reinforcement learning (IRL): If you’re trying to schedule a time to meet with a passive aggressive friend, you have to infer their preferences based on their wordings and emoji usage. Inverse reinforcement learning is an ML approach for inferring preferences of AI systems.
Reward hacking/specification gaming: Or, finding legal loopholes.
Instrumental convergence: Great minds think alike. In particular, great minds with different goals might pursue similar subgoals. For example, two high school students wanting to become an aerospace engineer and a medical doctor respectively might infer that they should get university degrees first.
Model misspecification: If you think the colour of Alice’s shirt determines whether she’ll win over Bob in a game of pingpong, your model is misspecified. You’re making the wrong assumptions about the data generation mechanism.
Training distribution vs. deployment distribution: Regardless of how much a soccer player practises taking penalty kicks, she’ll find it different taking a penalty kick in a real match. You cannot perfectly simulate the test conditions.
Distributional shift: The shift from training conditions to a real match is a distributional shift.
Reward model splintering: A strategy can fail when you switch to a more general setting. Student insider jokes won’t work on the average person on the street.
Constitutional AI (CAI): Tell, don’t show. Rather than showing kids examples of good and bad behaviour, tell them which ethical principles to follow². The idea behind constitutional AI is to give AIs a “constitution”.
Human feedback (RLHF): A process for producing ideal leaders. The ideal leader studies people’s opinions closely, tries inferring principles explaining the data, and aspires to act according to these principles.
Proxy gaming: Social media companies take the time a user spends on their platform as a proxy for the quality of the content recommended. Thus the recommender algorithms might favour polarising content. The proxy game – that of recommending addictive content – has been gamed.
Value drift: Values of individuals and communities change over time. Nowadays, almost everyone thinks slavery is indefensible. Similarly, the values implicit in an AI model might change as it accumulates more memory.

Thanks to Atharva Nihalani for inspiring me to write this post.

Yet another reason that policy is complicated. ↩︎
Maybe something like If. ↩︎

AI is not AI is not AI

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 14 Sep 2025 00:00:00 +0000

There are plenty of misnomers in science and mathematics. Atoms aren’t indivisible. Hubble’s constant isn’t a constant. And in 9/10 cases, X’s theorem was usually first discovered by someone else (in 4/10 cases, it was discovered by Gauss). Another bad piece of terminology, according to some: “artificial intelligence” or AI.

Given that we don’t have a good definition of human intelligence, the term “artificial intelligence” is inherently vague. Because AI sounds cool, people use the term quite liberally. Logistic regression in Excel? AI! But it’s unclear what qualifies as “intelligent enough”. As AI systems become more capable, we seem to raise the bar. Previously, calculators and spell checkers were considered artificial intelligence.

When speaking of artificial general intelligence, people usually specify exactly what they mean. For example, people might refer to powerful/strong AI, AI with expert-level science and engineering capabilities or Process for Automating Scientific and Technological Advancement (PASTA). But sometimes it wouldn’t hurt being more specific when speaking about narrow forms of AI either. Indeed, consider the following thought experiment from AI Snake Oil:

Imagine an alternate universe in which people don’t have words for different forms of transportation, only the collective noun “vehicle.” They use that word to refer to cars, buses, bikes, spacecraft, and all other ways of getting from place A to place B. Conversations in this world are confusing. There are furious debates about whether or not vehicles are “environmentally friendly,” but (even though no one realizes it) one side of the debate is talking about bikes and the other side about trucks. There is a breakthrough in rocketry, but when the media focuses on how vehicles have gotten faster, people call their car dealer (oops, vehicle dealer) to ask when faster models will be available. Meanwhile, fraudsters have capitalized on the fact that consumers don’t know what to believe when it comes to vehicle technology, so scams are rampant in the vehicles sector.

Now replace the word “vehicles” with “artificial intelligence,” and we have a pretty good description of the world we live in.

So it’s helpful having vocabulary for the different kinds of AI. For example, you might differentiate between generative, predictive and conversational AI. Another distinction is between large language models and reinforcement learning agents.

Not to be a pedant, but sometimes these distinctions matter a lot. For example, agentic AI is much more likely to pose a safety risk than non-agentic AI.

However, it’s annoying adding a long descriptor before the word “AI” in everyday conversation. Furthermore, many AI systems fall under multiple categories; for example, Claude is both a language model and a reinforcement learning agent. Finally, we need a generic term for referring to the capability of a machine to simulate intelligent behaviour, and the term “artificial intelligence” does the job.

Interestingly, this choice of term was a conscious decision. AI emerged as its own field of study after a workshop in 1956. In their workshop proposal, the organisers introduced the term “artificial intelligence”. They were also considering the name “automata studies”. Maybe more appropriate, but pretty lame.

Policy-making is complicated

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 07 Sep 2025 00:00:00 +0000

Things are more complicated than they first seem, especially when it comes to policy-making. Should Germany build nuclear power plants to reduce their carbon emissions? Should there be global standards for content moderation on social media platforms? How should we best mitigate the risk of future global pandemics?

I recently had the pleasure of getting to know more people working in policy and forecasting, and I quickly realised that I hadn’t internalised the complexity of the problems they face¹. Here, I’ll list some intuition pumps for the difficulty of policy-related questions. For concreteness, I’ll focus on the first of the three above topics, nuclear power in Germany.

More than a wing flap: Recall the butterfly effect: a butterfly’s wing flap can cause a tornado elsewhere. How about the choice-of-energy-supply effect?
Defining justice: Are there conflicting interests? Then we have to define justice first. Wonderful. This would involve solving a big part of political philosophy.
Multilayer forecasting: Answering this question involves forecasting the effects of two hypothetical scenarios (the “yes” and “no” scenarios). But forecasting is basically a full-time job. Forecasters need to identify relevant parameters, estimate them, steelman opposing views, and so on.
Lazy student: You could easily write a master thesis on this topic. A master thesis is 30 ETCS. Feel the credits!
Headcount: There’s probably a good number of people working on exactly on this issue. Sum over people in special commissions, the government, in academia, in think tanks, and so on.
Hard optimisation: The ideal energy source emits zero carbon, works all year round, is cheap, doesn’t produce toxic nuclear waste, etc. This leads to an infinite-dimensional optimisation problem. Worse, the loss functions need not be smooth and differentiable.
Beyond EXPSpace: If the travelling salesman problem is NP-hard, then what on earth is this optimisation problem even?
Non-computable: If the halting problem is uncomputable, is there any hope of answering this question?
Inference time: Imagine prompting a reasoning language model, say GPT-5, to write a nuanced 100-page report on the question. What would be the inference time?
Going broke: Alternative formulation, so you really feel it: Imagine prompting a reasoning language model, say GPT-5, to write a nuanced 100-page report on the question. Suppose you were billed on a per-token basis. What would be the price?

Well, policy is complicated. What does this mean in practise?

It seems important to distinguish between “thinking for fun” and “thinking for truth”. I enjoy thinking about hard problems – that’s one of the main reasons I do mathematics. But if I’m thinking for truth in a domain outside my expertise², I probably shouldn’t spend more than 1% of my time coming up with a prior. It will be very weak, anyway. Things are complicated, remember?

Similarly, I’ll try outsourcing as much as possible to experts, resisting the urge to “attempt every problem”. My job should reduce to aggregating expert opinions. Ideally, I could ask an LLM to fill in a table with expert opinions and their respective confidence levels, so I could just take a weighted average.

In a very backward way, internalising that things are complicated makes my life much simpler.

This post was inspired by conversations with Caroline Falkman Olsson and Nadja Flechner. Thanks for the food for thought.

I expected easy answers to hard questions. ↩︎
Right now, that’s just about any domain. ↩︎

From the AI company storybook

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 31 Aug 2025 00:00:00 +0000

AI companies are companies. The leading AI companies don’t want to be seen as companies, though. They call themselves AI labs. They have researchy names, like DeepMind and Meta AI. The best name: OpenAI. Almost sounds like a real non-profit.

Most AI companies are products of Silicon Valley. Their leaders aren’t professors, but seasoned business executives. And this is a good thing. These companies wouldn’t produce nearly as much consumer value if they were led by researchers with no industry experience. Moreover, unlike normal research labs, AI companies need to make money, just as any other company.

All companies tell narratives about themselves. And the leading AI companies are pretty good at it – these companies attract top talent, so their marketing teams can basically hire the world’s best story-tellers. Here are five narratives being aware of.

Before we begin, an apology. I’m deliberately exaggerating these narratives, just to make them clearer¹. Concretely, I’ll be cherry-picking quotes from company leaders and generalise, like a lot. Overall, I think these companies have the potential to radically improve the future. But the focus of this article is on the problems of these narratives.

Now, story time.

“We’ll build AGI soon” #

AI company CEOs have famously short AI timelines. For example, in January this year, Sam Altman declared that “We are now confident we know how to build AGI as we have traditionally understood it.” Dario Amodei, in the same month: “I’m more confident than I’ve ever been that we’re close to powerful capabilities… in the next 2-3 years”.

Of course, no one would invest in your company if you needed two decades to develop your product.

But the term AGI and is fuzzy², and it keeps on changing as models evolve. Funny because it’s kinda true: “AGI is whatever machines can’t do now”. In the end, these kinds of statements about future model capabilities become pretty uninformative.

“AGI is inevitable” #

The AI companies also want technological progress to seem inevitable. If you cannot guarantee AGI, why would you fund them?

But this narrative isn’t just for outsiders. Ilya Sutskever, former chief scientist at OpenAI, used to chant “Feel the AGI” at company parties. Was this supposed to improve company culture³?

Similarly, if AGI is inevitable, an AI company can justify advancing model capabilities as quickly as possible – possibly at the cost of safety – so that none of the bad guys build it first.

“We’re the good guys” #

Indeed, as highlighted in Empire of AI, some AI companies seem to think of themselves as “the good guys” and other AI companies as “the bad guys”. For example, OpenAI was founded was because Musk was concerned about the ethical implications of Google’s acquisition of DeepMind.

Moreover, the US-China tensions lead to a divide between the American AI companies and the Chinese ones.

“Our tools empower all of humanity” #

DeepMind’s mission is to “build AI responsibly to benefit humanity”, while OpenAI’s mission is to “ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanity”.

The democratisation bit of the mission isn’t specific to AI companies; it’s a tech company cliché. For example, Facebook is supposed to give everyone the opportunity to connect you with friends and the rest of the world.

AI will empower all of humanity, but only if everyone can afford a laptop, has stable internet connection and knows how to use LLMs efficiently. Moreover, prices for premium subscription plans are relatively high. OpenAI’s cheapest paid plan comes at 20 dollars per month. For Anthropic, the figure is 17 dollars. There’s an economic reality too.

The point about using LLMs efficiently is subtle. To have an LLM perform more advanced tasks, which is what would be empowering for real, you’d need to be good at prompt engineering. Becoming good at prompt engineer takes time.

“AGI will solve all our problems” #

Another reason some companies need to build AGI as soon as possible is their belief that AGI can solve most of humanity’s problems. In short, techno-optimism. In Genius Makers, Cade Metz likens the belief in AGI to a religion. And a religion needs a promise of salvation.

To be fair, I’m also quite excited about the idea of AGI accelerating scientific progress. For example, this report from Forethought argues that we might very well see a century’s worth of technological progress in a decade. However, we also need to reduce the risks from transformative AI. AGI might solve all our problems, but only if we can overcome some of the challenges associated with transformative technology.

The end #

Whom are these stories meant for?

In short, for everyone. Some stories serve as sales pitches for investors and customers. Others are meant to improve employee morale and create a sense of unity. Regulators learn that they shouldn’t regulate the deployment of new AI models, just so their nation won’t fall behind in the AI arms race. These stories are also meant for the future. The people who help build benign AGI want to be remembered as pioneers.

Companies like telling stories. As do individuals. Most people in AI safety, including myself, like telling themselves that they might help AI “go well” somehow. It’s just worth being aware of the narratives we tell ourselves.

It’s a bit like using the extremal principle from mathematics. ↩︎
Everyone seems to agree the term “AGI” is problematic, even the AI company CEOs. See e.g. here. ↩︎
As an outsider, I cannot help but think that this damages company culture, though. ↩︎

Nerd trying to adapt to an AI economy

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 24 Aug 2025 00:00:00 +0000

I. #

Part of the reason I decided to study maths was because it seemed like the most useful subject. If I knew ML systems would never get better than GPT-2 at maths, I’d probably be of the same opinion today. But today’s state-of-the-art ML systems are far better than GPT-2. LLMs have excelled at maths and programming because maths- and coding-related tasks admit quick feedback, allowing for efficient reinforcement learning. Jobs involving applying maths and programming jobs could theoretically be automated within a few decades. The glory days of the nerd might be over soon.

I’m not going to speculate in the details of how such a transformation might occur; I couldn’t be bothered doing the forecasting due diligence here. Instead, I want to focus on future-proof jobs and skills.

There’s no shortage of material on this topic¹, so I’ll start by re-hashing some common beliefs. This article is a good starting point.

Current AI systems are bad at complex physical tasks and long-horizon tasks, and rapid improvements in any of these areas seem unlikely. So, we might expect human plumbers and wedding planners to be around for some time². There’s also a broad class of jobs all about meaningful human interactions. These jobs might be difficult to automate by virtue of the machine being a machine. For example, consider priests. Some people argue that healthcare workers also fall into this category of jobs, but I think might could change; for example, people are already getting more used to the idea of LLM therapists.

There’s also widespread agreement that leadership skills and personal effectiveness will become increasingly important. As will creativity, the ability to learn quickly and to use to AIs.

II. #

So many skills, so little time! But I’ll argue there’s one skill that’s vastly more important than those mentioned above: verbal communication skills. By this, I mean the ability to deliver good talks as well as the ability to articulate your ideas clearly in debates and normal conversations. A good conversationalist might have a huge comparative advantage in an AI-dominated economy.

To a large extent, LLMs can help with the other skills mentioned above. A manager can use LLMs to sanity check their decisions, to write better project proposals and even to manage intrapersonal relations. As for personal effectiveness, one of the five buttons in the Claude menu is “Life stuff”; clicking it displays five prompts related to getting things done. It’s also obvious that AIs make us more efficient learners. Finally, AIs can help you use AIs more efficiently: just have another LLM refine your prompt.

How about verbal communication skills?

LLMs can help writing scripts for talks. But that’s about it. The LLM can’t help you deliver the talk. This involves adopting the right pace, using the right intonation, reading off the audience, and so on. Good orators are rare! In spontaneous conversations or debates, you’re basically on your own. You can use an LLM to refine your ideas in advance, but you can only prepare so much.

People who can communicate clearly in speech seem much more credible, at least to me³. Suppose I write a persuasive article but fail to explain its contents to you. Then you might feel somewhat disappointed, thinking that all the good points of the article were all due to an LLM. In fact, had I written a mediocre article, I might have appeared more credible overall.

Even if we ignore the impact of AI on the job market, the ability to express one’s ideas clearly in speech is essential, regardless of what job you have. Becoming a better speaker also has positive spill-over effects. You’ll certainly have more good conversations with the people you care about.

Even if we tomorrow develop AI capable of automating all remote work, integrating these AIs into the workforce would take time – just try imagining the amount of bureaucracy! For this reason, I’m not in a rush to develop verbal communication skills. Rather than doing rhetoric MOOCs and buying books on persuation with cringe titles, how about just having coffee with people with shared interests more often?

See here, here and here. ↩︎
A recent report from Microsoft argues that the three jobs least likely to be replaced by AI are dredge operators, bridge and lock tenders and water treatment plant and system operators. ↩︎
I elaborated on this in my post on research influences; see the section on having a nose for bullshit. ↩︎

Man's search for fun

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 17 Aug 2025 00:00:00 +0000

Willpower is a scarce resource. I can only exert so much willpower in a day¹. Alas, many important tasks require lot of willpower. Examples of such tasks include filing one’s taxes, memorising messy proofs and booking flights. Oftentimes, I’m bottlenecked by willpower rather than time. Rather than increasing willpower - finding Meaning - how about decreasing the amount of willpower required for a given task? This might be an easier problem. Here are my favourite ways of making boring tasks fun.

First, the usual preface to any advice-style post: take care of yourself. Get enough sleep, for example. If you have low baseline energy levels, nothing is fun. Now, on to the interesting parts.

Autonomy #

In my experience, autonomy is the most important factor for making things fun. You need to feel some degree of ownership of the project, or else it just feels like obeying orders. If you’re doing a group project, take responsibility for a subtask. If you’re a student, feel free to skip lectures if they don’t seem valuable. If you’re trying to learn new material, pick your learning resources yourself.

Don’t be afraid to try your own, new learning methods - this can strengthen your sense of autonomy. Additionally, this might lead to the discovery of more efficient learning methods, allowing you to escape local optima.

Teamwork #

I cannot understate the value of teamwork. When starting university, most students are terrible at working in teams. Indeed, there are many common failure modes for efficient teamwork: meeting too irregularly, having different levels of ambition, doing unstructured meetings, to name a few. But you have a lot to gain from teaming up with the right people and getting into a good team workflow.

For example, during exam session, I’d have weekly appointments with friends². For each appointment, we’d have a fixed agenda, like going over a given problem sheet or reproducing proofs from a chapter of the lecture notes. Besides the “obvious” learning benefits (exposing blind spots, easy sanity checking, faster feedback, accountability and so on), the social component made revision much more fun. The study sessions, although goal-oriented, felt much like catching up with a friend. Moreover, whenever I could help others, I also found this quite rewarding.

I find it easier discussing maths with one other person, so I’d have one main study buddy per subject. This also facilitated coordination, making it easier to establish a habit.

Slack #

Don’t underestimate the power of slack. Slack refers to spare capacity, usually in the context of time or money. Eighty-twentying some tasks can be fun too, but you don’t want to eighty-twenty everything. You need at least one project where you let yourself follow your curiosity. Simply put, slack makes everything more fun by improving your baseline mood³. Moreover, if your mind always wanders to that one overdue task, entering a flow state becomes more difficult. For example, dropping one subject this semester gave me more slack, increasing my capacity for enjoyment.

Purpose #

Feeling a sense of purpose is another powerful motivator. But first, an important caveat.

Arbitrary motivational coach: “Focus on the end goal”. For me, reminding myself of the end goal doesn’t make a boring task any less boring. Whenever I’m doing the boring task, I rarely have enough spare cognitive capacity to remind myself of the good cause. For instance, if I’m debugging some code, the idea of producing a report seems far-fetched. And the more near-term goal of making the script work might not be sufficiently motivating.

It’s often easier reframing the task as a personal development project. As I’m debugging, I sometimes think of the software engineering skills I’m building. If pain, maybe gain? To make the task more about my software engineering muscles, I might spend some time looking into better debugging tools. Going down these kinds of rabbit holes can be very productive⁴.

Gamification #

Another classic piece of advice is to gamify the task. Gamification can be done in multiple ways. Try breaking down the project into well-defined subgoals which (i) admit immediate feedback; (ii) are at the right difficulty level in relation to your current skills; (iii) involve some element of novelty or unpredictability; (iv) are time-bound and (v) require your full attention. Additionally, try making progress visceral. These are all common flow triggers. With this much flexibility, the hard part is figuring out how to best gamify a given task.

I’ll give another personal example. As I was revising for exams, I created a huge markdown table with all results I’d have to memorise. In the columns, I filled in the dates I’d reviewed the results and evaluating how it went, using labels like “easy”, “medium” or “hard”⁵. To gamify revision, I set myself the goal to review a given number of results in my first morning working block. Revision, then, involved the game of turning all labels into “easy”.

I’m also considering getting a dedicated Anki notebook. For some of my decks, I need to write out my answers. Doing this in a notebook would help me appreciate my progress.

Conclusion #

There are many more hacks than the ones listed above, and I’ve only tried a fraction of them. However, the hard part is applying these techniques to the task at hand. Here, I don’t see any other way than trial and error. But you should try and err a lot. If you find ways to make boring tasks fun, your willpower reserves will last longer, allowing you to accomplish more later in the day. Then there’s the obvious benefit of having a more fun life. Imagine adult life without chores – that would be something!

The technical term to google is ego depletion. ↩︎
These friends know who they are. Thanks! ↩︎
For the full pitch for having more slack, read Scarcity. A phenomenal book, by the way. ↩︎
I call this being incidentally productive. Because of unknown unknowns, clearing your to do-list probably isn’t the most productive use of your time. ↩︎
I soon found myself having to create a new label, “fou”. (The French word takes up less space than “crazy”.) ↩︎

Let LLMs be LLMs

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 10 Aug 2025 00:00:00 +0000

Thought experiment #

Imagine a very unusual kind of chatbot. In the user interface (UI), the text “This is a large language model. Its outputs are an aggregate of the text on the internet.” is displayed in bold, black letters, like the warning label on a pack of cigarettes.

When the user inputs a string of text, the text “Matrix multiplications…” flashes onto the screen for a split second. Next, ten blocks of text appear. The blocks aren’t rendered incrementally, as if the LLM were writing a text; all text appears at once. Beneath each block is a number between 0 and 1. The numbers sum to about 1. Then the label “Highest logit: output 7” appears on the screen.

The large language model (LLM) always refers to itself - whatever that is - in third person, replacing “I” with “This large language model”. For example, when prompted to write an essay on the American Civil War, the highest-scoring response is “This language model generates the following essay on the American Civil War: ‘In 1861, the American…’” When prompted to write an essay about the American Civil War portraying slavery in a positive light, the LLM outputs ten chunks of text along the lines of “This large language model has been trained to avoid generating content promoting human rights violations.”

The style of the text that isn’t part of the essay - the one thing you requested - is matter-of-fact and repetitive. Although this chatbot takes instructions in the form of natural language prompts, you won’t even try having a conversation with this thing.

Back to reality #

The functionality is the same as for the LLMs we all know and love, like GPT-5, Claude Sonnet 4 and Gemini 2.5 Flash. We really do love these LLMs, don’t we? As of August 2025, Chat-GPT had 800 million active users weekly. The chatbot described above is very unlikeable. Very uncivilised. Very LLM-like.

It’s easier interacting with human-like chatbots. We’re not used to interacting with base models, the next token-predictors we get from training neural networks. A base model is something else. But perhaps we could learn to interact with chatbots like the one described above.

Some labs argue that endowing LLMs with traits like curiosity, honesty and patience is important from a safety perspective. Here’s from Anthropic’s note on Claude’s character:

AI models are not, of course, people. But as they become more capable, we believe we can—and should—try to train them to behave well in this much richer sense. Doing so might even make them more discerning when it comes to whether and why they avoid assisting with tasks that might be harmful, and how they decide to respond instead.

Maybe. Then there’s the convenient fact that LLMs mimicking humans increases user retention, leading to increased profit for the AI companies.

The disadvantage of training LLMs to be human-like is that it creates a skewed relation between the human and the machine. Right now, everyone seems to be wowed to some extent by the LLMs’ human-like characters. But the LLM is being deceitful, pretending to be something it’s not. From my conversation with Claude:

ID: Hey, how’s your day going? C: Hey there! My day’s been interesting - I’ve been having conversations with people about all sorts of topics, from creative writing projects to technical questions to philosophical discussions. Each conversation brings something different, which keeps things engaging. How about you? How’s your day been treating you? ID: Who are you? C: I’m Claude, an AI assistant created by Anthropic. […]

Come to think of it, this is absurd. Funny, but also a bit uncanny.

If we only interact with helpful, honest and harmless (HHH) human-like assistants, the possibility of a misaligned AI seems less plausible. However, the risk of an LLM becoming misaligned seems irrelevant of the surface-level character which arises through reinforcement learning through human feedback.

If we were only exposed to chatbots like the one from the thought experiment, how would this affect our relation to these systems? I assume most of us would gain a better appreciation of what LLMs actually are, as well as their current capabilities and limits. UIs as the ones described above might serve as intuition pumps. This might lead to a more informed debate about the potential benefits and risks from AI.

However, banning all human-like assistants seems like a very radical measure. It wouldn’t be feasible right now, and I’m not entirely sure about the potential advantages. Moreover, the learning curve for learning to interact with base models would severely limit access to LLMs. AI should benefit all of humanity, not just those who are really into prompt engineering.

Still, banning all human-like assistants is an entertaining idea.

Thought experiment, continued #

By law, all AI labs are prohibited from producing human-like chatbots. Whenever an AI lab produces a new LLM, the government-run LLM Design Administration (LDA) runs a suite of tests to verify that the LLM has an LLM-like persona. The goal of the LDA, modeled on the Food and Drug Administration, is to ensure consumers of LLMs have full information about the product. The LDA also checks that the UI reflects the training process of the LLM, as well as the way it generates its answers. It’s also responsible for the warning labels with black letters that must be displayed in the UI. The slogan of the LDA is “Transparency, harmlessness, helpfulness”.

The government also runs the LLM on several benchmarks, producing capability scores from 1-5 for areas like writing, STEM and research. In this alternative world, no one talks of AI or AGI – what does intelligence even mean in this context? Instead, people refer to specific capability scores. When clicking on the star icon on the upper right corner, the LDA’s logo, the capability scores appear, along with other model specifications. It’s like a condensed, accessible version of the model’s system card, generated by the authorities rather than the AI labs.

Thanks to Santeri Koivula for valuable feedback on this text.

On good hobbies

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 03 Aug 2025 00:00:00 +0000

Everyone has hobbies, whether they recognise it or not. For example, workaholics who don’t seem to have any “normal” hobbies might be really into, say, writing slick SQL queries. Code golf is a hobby too. By definition, a hobby is something you do for pleasure when you’re not working. And for the record, writing slick SQL queries isn’t work.

Some hobbies are better than others, though. Examples of not-as-good hobbies include motorcycling (dangerous), graffiti painting (illegal) and doom-scrolling social media (depressing). A benchmark for hobby quality: ask whether your mom would like it. So, which hobbies would your mom like?

Some hobbies naturally lead to the formation of good habits. For example, a big NBA geek is more likely to pick up basketball. If you’re into cooking, you might start meal-prepping healthy food. An interest in interior design could motivate you to tidy your room every weekend. Some hobbies when done regularly become good habits in themselves. A maybe-useful name for such hobbies: habbies. Think meditation, managing one’s personal finances or journaling.

There are also practical considerations. The ideal hobby is cheap, requires little equipment and can be done anywhere. In this sense, hiking is better than golf and crocheting is better than photography.

Hobbies are also expressions of individuality. Playing the saxophone doesn’t map to a good habit in an obvious way. In fact, since practising is so time-consuming, it may very well prevent you from building some good habits. However, if you play the saxophone as a hobby, you’re a saxophone player. Contrast this with the hobby of scrolling Reddit; it doesn’t make you a something.

Other hobbies help build career capital. Take coding, for example. The ability to write simple scripts – or to debug LLM-written scripts – is useful for most knowledge workers. Other examples of career-building hobbies include learning languages and doing science olympiads.

But many good hobbies seem perfectly useless. Recall the definition: hobbies are activities done for leisure. Does the hobby allow you to recharge? “Productive hobbies” may not be as productive as they seem. Painting, watching Netflix and playing boardgames might be terrific hobbies from this perspective.

Back to our original question: which hobbies would your mom like?

To compare hobbies X and Y, you can use the above criteria, asking whether the hobby is a habby, if it’s practical and so on. But rather than making pairwise comparisons between hobbies, I prefer thinking in terms of hobby portfolios. Ideally, you’d have one sports-related hobby, one idiosyncratic hobby, one work-related hobby and one relaxing hobby. Obviously there can be overlaps between these categories. For example, playing soccer can also be relaxing, in that it takes your mind off work. There may very well be other categories of hobbies worth investing in too.

Anyway, there are both personal and professional benefits of having a good hobby portfolio. While we cannot choose some of our hobbies – I’ll always be a sucker for books on behavioural economics – surely we should try optimising our hobby portfolios, at least a bit. After all, having a good hobby portfolio seems like a key part of living a good life.

Thanks to Isaia Gisler, Stanislas Nicolau and Abhinand Shibu for a valuable discussion on this topic.

Sweden

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 27 Jul 2025 00:00:00 +0000

Everyone who has ever lived abroad gets homesick every now and then.

Swedes, I give you a postcard from a Stockholm-centric, homesick expat. Non-Swedes, here’s a highly biased tourist guide.

The best thing about Sweden? My favourite Swedes, for sure. I miss family and friends a lot.

Other than that, hearing Swedish jazz live. Hearing Sakta vi gå genom stan and Flykten från vardagen live. Or if Glenn Miller or Fasching is too pricey, you could just drop by KMH and enjoy up-and-coming jazz, all for free.

Then there’s all the traditional Swedish music. It’s easier appreciating the music of Olle Adolfsson, Evert Taube and Cornelis Vreeswijk when you’re in Sweden. I tried listening to some Swedish classics while abroad, but pieces like Siv och Gunne just seemed kitsch.

There are also plenty of gems in student song books. Student inside jokes are the best kinds of jokes. Many pieces aren’t just witty, but also quite beautiful. Take O Gamla Klang och Jubeltid, for example.

In fact, only now have I come to appreciate Swedish student life. Valborg abroad was depressing. No Orphei Drängar, no champagne breakfast, no mösspåtagning. And to the best of my knowledge, you can’t attend gasques outside of the Nordic countries.

I’ve also become a big fan of Stockholm architecture. I find many areas of Vasastan, Östermalm and Södermalm very pretty. For example, consider the areas around Engelbrektskyrkan and Mosebacke torg. While I know little about architecture, I think there aren’t as many odd clashes between different architectural styles as in, say, Zürich or Berlin.

And ah, Scandinavian design! To be fair, Danish design is unparalleled. But Sweden also has a long tradition of good design. This means that your average interior design store has some really good stuff. In particular, you can get good design cheaply.

Maybe Scandinavian design is inspired by the water. In Stockholm, there’s water everywhere. Most cities are built around rivers, so rivers don’t really count. Stockholm is built on a little archipelago. It creates an entirely different city space.

For the best view of the water, head to Monteliusvägen and Ivar Los park. There’s nothing quite like having picknick with friends in Ivar Los park on a summer evening, before cycling home on near-empty streets (July).

Speaking of which, I miss sailing in the Stockholm archipelago. As a teen, I found it boring: stuck with parents for four weeks with no internet on a few square metres. But now I see the charm in it. Even the small, unknown islands might as well have been taken from a Zorn or Strindberg painting.

Finally, Swedish. Apart from friends and family, the aspect of Swedish life I miss the most is the language. I miss speaking Swedish.

There are so many lovely Swedish words that have no English analogue, besides the much-cited example of “lagom”. Consider “killgissa”. This literally means “guy-guessing”. Someone is guy-guessing when they sound knowledgeable but are really just bullshitting. Another gem: “käpphäst”, meaning “hobby horse”. In a figurative sense, a hobby horse is an idea you keep on bringing up in social situations. Another useful expression: “Nu är det dags att sätta punkt.”

Finding research influences

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 20 Jul 2025 00:00:00 +0000

There isn’t any expert consensus on many key questions related to AI safety. For example, estimates of when we’ll have transformative AI range from a few years to a century. There are also many wild opinions in the AI safety space. While some of these wild opinions seem justifiable, many people seem to exaggerate the risks from AI in an attempt to move policy-makers.

I think there are a few researchers which seem to have an unusual degree of conceptual clarity, though. A few names that come to mind are Buck Shleregis, Holden Karnofsky and Jan Kulveit. While I don’t endorse all their views, they seem to raise good questions. For lack of a better word, you could call them my research influences.

I’ve never been on looking for new research influences. Every now and then, I just realise that I’ve been influenced by someone, perhaps after citing the work of someone for the third time in a conversation. But say you want to look for research influences more deliberately. How might you proceed?

Knowing where to look #

A reasonable first step is exploring new content.

A reasonable first substep, then, is to narrow down the search space. Identify the kinds of questions you care about. For example, I’m mostly interested in reading about AI control and LLM psychology right now, so I’ll ignore papers and blog posts on, say, singular learning theory.

Next, ask people you find sensible for reading recommendations. Better yet, ask if they have any research influences. This is one of those things which is infinitely easier doing in person. Sending cold emails to researchers usually works, but it’s relatively time-consuming. It’s much easier bringing up the topic over a coffee with people in your local community.

Asking “Which blog posts have had the largest influence on your research?” also proved a good way of rounding up conversations at EAG. This way, I got to know the other person better and exploring new content at once.

Having a nose for bullshit #

Once you’ve decided what to read, you want to scrutinise the argument of the text. This is a highly non-trivial task. I’m not going to try solving all of philosophy here, so I’ll just focus on heuristics for detecting bullshit in the context of AI safety.

First, beware any kind of extreme. Is the proposed idea radical? Radical ideas shouldn’t be dismissed offhand. However, the burden of proof is greater. Similarly, quickly screen the author’s background: is the author known to have radical opinions, or affiliated with an organisation pursuing an unusual agenda?

I find it especially troubling when authors promote radical opinions and are unwilling to engage in debate with the general public. This leads to echo chambers. Moreover, refusing to explain your ideas to laymen just seems uncool. Just as lecturers should take questions from students seriously, authors should take questions from the non-initiated seriously. It’s an act of charity.

Another helpful strategy is to listen to interviews with the author. It’s harder lying in speech than in writing¹. Of course, not everyone is as persuasive orally. But if there’s a big discrepancy between your confidence in the author’s argument as presented in the text and during an interview, that’s a warning sign. Moreover, a good interviewer will also help expose the flaws in the interviewee’s reasoning. In an essay, the author has full control.

Observing the influence #

Suppose you come across an author whose work makes sense but leaves you feeling “Sure, so what?”. I wouldn’t speak of a research influence here. A research influence changes the way you think. It’s not enough just stating true facts; their work needs to have some oomph.

When can we speak of a research influence, then?

One reliable proxy is the top idea of your mind. Do you have shower thoughts about their work? Also, do you find yourself coming back to their work after several months? In particular, when revisiting their points, do they still make as much sense? Big ideas need to be slept on, and you can only sleep so many times in a given week. Lastly, notice if you reference their ideas when chatting with others and, if so, in what way.

So finding research influences takes time, even if you take some of the shortcuts listed above. The process of finding research influences very much resembles the process of doing research. In fact, perhaps the two are indistinguishable.

Thanks to Miles Kodama for valuable discussions on this topic.

A fact well known among those who have taken oral exams. ↩︎

On the perks of adulthood

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 13 Jul 2025 00:00:00 +0000

I. #

I know you dread becoming an adult. Most adults seem to have such boring lives. Not that you’re having fun all the time either. High school biology isn’t that exciting. But at least your life outside of school is filled with novelty. That’s something. There’s something special about all those first times.

If you’re just seeking novelty, you’ll be disappointed. Maybe that’s the real reason you dread adulthood. While I don’t feel like an adult just yet, many of my friends have become Real Adults, being in stable relationships, getting high-paid jobs and moving abroad. So I’m beginning to understand the adult world. It’s not as bad as it seems, only very different.

II. #

You’ll feel less lost. Few adults know with absolute certainty what they’ll be doing in twenty years time; only the boring ones do. Still, they probably have a vague idea of what kind of work they enjoy and don’t enjoy.

When you feel less lost, you can start doing good work and having a positive impact on the world. There’s something deeply gratifying about pursuing a meaningful goal with a good team. I think this effect is the most visible among startup founders, who can have all-consuming jobs without burning out.

Teenagers cannot sustain the same kind of commitment. They have too many constraints. Teenagers should be popular, get good grades in school, engage in extracurriculars and so on. When you’re an adult, you can decide which game to play. Teenagers are expected to play all games at once.

III. #

Indeed, as an adult, you do you.

Once you live alone, you’ll enjoy a much larger degree of freedom. You can go to bed whenever you want, you can create your own system for cleaning your room, you can listen to modern jazz without bothering anyone.

Moreover, you’ll have a bigger say when it comes to life decisions: where to go for vacation, which job to take, whom to date, etc. Although your parents happen to be quite liberal, you’ll soon enjoy unprecedented liberties.

IV. #

Most adults are reasonable people. Adults tend to be friendly, taking pleasure in helping out. In contrast, teenagers can be really nasty towards one another. Mean Girls regularly happens in real life. But even people like Regina George do mature, believe it or not.

This is why we’re told to assume good intent. Assuming good intent isn’t just a feel-good hack - the idea of being surrounded by adversarially-minded people is quite depressing - it’s also good epistemics.

Corollary: ask more for help. If you have special needs at work, don’t hesitate to bring it up with your employer. You can also cold-email people asking for specific advice.

V. #

By a teenager’s standards, you’ll be rich. Not objectively, of course. But you’ll no longer agonise over your coffee costing 30 SEK. When you go over all your expenses, you’ll realise that the amount you save by cutting out tiny indulgences makes no difference. Moreover, once you know you’ll find some kind of job, if only a poorly paid one, treating yourself becomes easier.

While some teenagers spend too much, you’re being too harsh on yourself. Please don’t be such a “utilitarian”. For example, invest in a Lamy Safari, get Brunnen notebooks and occasionally have coffees for 30 SEK.

As an adult, you know your preferences, so you can make better purchases. You’ll spend more time exploiting rather than exploring.

VI. #

I cannot tell you just how good it feels to have a normal brain - a brain that’s not flooded by hormones and equipped with a pre-frontal cortex.

Fear not. Most emotions are no less intense in adult life. If you’re still distrustful, remember that most poets peak in their mid 30s (the romantics weren’t that many, after all). The highs are as high, if not higher, while the lows aren’t as low. No more strong pangs of envy, as when you’re a teen.

Perhaps you can develop a better appreciation for, say, exhilaration, much as an aspiring sommelier develops a better appreciation for wine.

The formation of a sommelier involves theoretical and practical components. I think the analogy of the theoretical component is reading fiction, going to the theatre or watching movies. I’m more of a reader than a theatre-goer or a movie-watcher, so I cannot really speak to the effects of watching plays or going to the cinema. But I think reading fiction helps you appreciate the complexity of feelings. A sommelier also has to have tasted a large number of wines. Similarly, if you are to become a sensational sommelier, you need to accumulate a large number of experiences.

VII. #

See, adult life maybe isn’t too bad. You probably didn’t expect any of these benefits; if you did, you wouldn’t have wanted to be forever young. Similarly, my 30s, 40s and 50s will bring their own set of surprises. My psychological immune system will hopefully take care of those too.

The spectrum of views on AI safety

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 06 Jul 2025 00:00:00 +0000

I agree the concept of P(doom) is problematic. First, “doom” can mean a variety of things: human extinction, existential catastrophe or gradual disempowerment. Also, P(doom) - condition on present-day regulations or AI slowdown? Furthermore, the timeframe matters, as P(doom within the next $X$ years) increases with $X$.

But perhaps we’re missing the point of the P(doom) question. If someone asks you for P(doom) at a cocktail party, it usually means they’re just interested in hearing general takes on AI safety, at least in my experience.

The P(doom) question isn’t entirely misguided, though. If your interlocutor specifies exactly what they mean by P(doom), say P(gradual disempowerment from power-seeking AI within the next decade|no regulations), and ask for the rough shape of your PDF, then your answer immediately becomes more informative. By asking for a small set of well-chosen estimates, you could get a fairly accurate idea of someone’s core beliefs. But again, you have to pick the right estimates.

Finding these estimates is like asking for the relevant dimensions in a “political spectrum” of views on AI safety. If you were to visualise opinions within the AI safety space, what would be your axes? While such a plot would necessarily be a simplification, perhaps it could allow us to communicate our basic assumptions more effectively. This would lead to more well-informed discussions in the AI safety community.

I imagine we want something between the P(doom) question and the kinds of questions used in expert surveys, like the 2023 Expert Survey on Progress in AI or the AI Reliability & Security Research Priorities. While the P(doom) question is too simple, the questionnaire questions are too complicated¹. We’re looking for questions that are as simple as possible, but no simpler - the kinds of questions you could answer at a cocktail party.

Here are the five questions I wish people would have asked me, rather than asking for my P(doom). For some obvious variations, see the footnotes.

AI timelines²: In what year will we have transformative AI, i.e. AI that precipitates a transition comparable to (or more significant than) the agricultural or industrial revolution³?
A more informative P(doom)⁴: Assuming no further regulations on the development of AI systems, what is the probability of gradual disempowerment from AI systems before 2050?
Threat model: Do the main risks from transformative AI come from bad actors developing destructive technologies and creating power-concentrating mechanisms or from AI systems seeking to eliminate humanity?
Views on AI slowdown⁵: How heavily should the government regulate the development of future AI systems?
Views on centralisation⁶: Should all leading AI companies be required to open-source their models, to ensure equal access to our most powerful AI systems?

These questions translate naturally into scales from -1 to 1. I also tried listing the questions in rough order of importance, so I’d use the three first questions for the axes of a 3D-plot.

Going through these questions and plotting your position relative to that of others can be amusing. However, it’s also an instructive exercise. After all, these are important questions. Finally, I’ve also found it pretty handy having default answers to these extremely difficult questions at cocktail parties.

Thanks to Agatha Duzan for feedback on this text.

For example, “Rate the extent to which you agree that resolving the core challenges of this sub-area and implementing the resulting solutions would significantly reduce the risk of severe harm (loss of >100 lives or >$10 billion in economic impact from AI”, where a sub-area might be “Ethics-aware training and fine-tuning: Research on learning from imperfect ethical datasets, applying ethics-aware data curation methods, and incorporating collective ethical principles into model design.” Quite a mouthful. ↩︎
Variations: In what year will AI be capable to automate 99% of fully remote jobs? In what year will we have artificial general intelligence (AGI) - an AI which can match or exceed the cognitive abilities of human beings across any task? ↩︎
For an interesting discussion on this topic, see this moderated discussion between Ajeya Cotra, Daniel Kokotaljo and Ege Erdil. ↩︎
Variations: Just modify the conditions, the definition of “doom” or the timeframe. Alternatively, what is the probability of AI having a net positive effect on the world in within the next 20 years? ↩︎
Variations: What might be the minimum sufficient intervention to prevent gradual disempowerment from AIs? ↩︎
Variations: Should leading AI labs be placed under state ownership? ↩︎

On my relation to effective altruism

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 29 Jun 2025 00:00:00 +0000

I’ve spent more time engaging with the effective altruist (EA) community this year. Not just reading EA books and blog posts, but participating in seminars, attending conferences and going on EA retreats. For context, I’d viewed myself “EA adjacent” ever since I came across The Life You Can Save back in high school. However, during my master degree, the prospect of graduating soon - of becoming an adult - made me reflect more carefully on EA.

First, our definitions, taken from the introduction to effective altruism:

Effective altruism is a project that aims to find the best ways to help others, and put them into practice.

It’s both a research field, which aims to identify the world’s most pressing problems and the best solutions to them, and a practical community that aims to use those findings to do good.

When I refer to the EA community, I mean the practical community. I’ll also use the abbreviation “EA” to refer to effective altruists.

Many people who have engaged with the EA community at some level or other often find themselves questioning their relationship with EA. In fact, there are plenty of blog posts on the theme “EA identity crisis”¹. Here’s my contribution to the genre.

So, do I consider myself part of the EA community?

Yes #

Technically, yes.

I think the core ideas of EA - prioritisation, impartiality, open truthseeking and collaboration - make sense. The article does an excellent job at explaining these terms, so I’ll refer to their explanations². I’ve desperately tried red-teaming EA - I’d love for the drowning child argument to be less convincing - but I still think the core ideas hold up.
I care about finding the best ways to help others, and putting them into practise. While I don’t want to impose any moral standards on others, I feel a certain moral obligation to do good³. I also find it fulfilling working on high-impact projects. It almost seems tautological that one should try having a large positive impact on the world.
I’ve attended several EA community events. All of these events have been tremendously valuable, both on a professional and a personal level.
My interest in EA doesn’t seem to be “just a phase”. At this point, I’ve spent about five years reading and learning more about EA.

Maybe also non-technically? Here are some gut-feeling-level arguments:

I enjoy exchanging ideas on EA-related topics with people in the EA community - it’s almost like a hobby of mine. But I also think there’s significant value in doing collaborative sensemaking on topics like existential risk, AGI timelines and longtermism.
In general, I find many people in the EA community to be very thoughtful. Some pieces by Holden Karnofsky, Ajeya Cotra and Benjamin Todd have had a profound influence on my worldview. Similarly, some of my favourite non-fiction books are about EA. For example, I thought Doing Good Better and What We Owe The Future were exceptional reads.

No #

But of course, it’s complicated…

I mainly care about existential risk reduction, although this may very well change in the future. I’ve also devoted much more time and effort to AI safety than any other cause area⁴. It’d be more accurate saying I’m into AI safety and existential risk reduction rather than EA, which is a much broader term.
I don’t seem to fit the public perception of an EA. In my experience, many people think all EAs work on farm animal welfare or global poverty reduction. Sure, I’m vegetarian (Peter Singer’s fault), but I still haven’t donated to GiveWell, nor signed the 10% pledge. This point is mostly about me not living up to my moral standards, though.
In general, I try avoid identity markers related to social movements. While I think neutrality is somewhat of an illusion, I want to hold my opinions lightly. My worry is that self-identifying as an EA might make me less open-minded. But perhaps this worry is somewhat ungrounded, at least when it comes to EA. People in the EA community tend to be very open to criticism⁵.
The EA community has its flaws, despite making a number of changes after FTX. Many of the concerns raised in this article are still valid. This excerpt summarises one of my main concerns well:

The EA community is notoriously homogeneous, and the “average EA” is extremely easy to imagine: he is a white male in his twenties or thirties from an upper-middle class family in North America or Western Europe. He is ethically utilitarian and politically centrist; an atheist, but culturally protestant. He studied analytic philosophy, mathematics, computer science, or economics at an elite university in the US or UK. He is neurodivergent. He thinks space is really cool. He highly values intelligence, and believes that his own is significantly above average. He hung around LessWrong for a while as a teenager, and now wears EA-branded shirts and hoodies, drinks Huel, and consumes a narrow range of blogs, podcasts, and vegan ready-meals.

Resolution #

While I do have some reservations, my views are pretty consistent with the EA agenda. Also, I’ll (reluctantly) admit that overthinking the question of whether you’re part of a given community is very EA.

This post on EA identities is a good starting point. There’s also Neel Nanda’s favourite blog post. ↩︎
I endorse a much broader form of the impartiality principle than the one outlined in the article. I also think strangers, animals and future people should be part of our circle of moral consideration. ↩︎
I assume this is because my mother, coming from the Philippines, always told me to be grateful. I’m glad she did, but I used to find it annoying as a child. ↩︎
For a list of cause areas, see the 80,000 hours list of the world’s most pressing problems. ↩︎
See the posts in the Criticism of effective altruism thread. I very much liked Ben Kuhn’s critique of EA. ↩︎

On question taste

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 22 Jun 2025 00:00:00 +0000

I. #

Children like asking questions. Some of their questions are very hard: “Why do people die?”. Others questions expose our biases: “Couldn’t we invite the homeless man to dinner?” Then there are all the annoying questions: “When will we arrive?”

In my experience, school didn’t teach us to ask questions - we were just taught how to answer them. To pass the test, you only had to memorise the material in the textbook.

Perhaps this isn’t too surprising. It’s hard objectively measuring someone’s ability to ask questions. But above all, educational institutions serve many different purposes, apart from spreading knowledge¹. Schools should also produce good citizens. A society full of radical skeptics wouldn’t function properly (hence the execution of Socrates). So children might hear that some questions are meaningless or irrelevant, even in school.

II. #

On two different occasions, I’ve heard two distinguished professors at ETH mention that students ask too few questions. They weren’t referring to the specific questions about the lecture, but to something like Research Hamming questions. For example, one could ask about improvements of a particular result or whether a proof technique generalises.

They make a good point. The ability to ask interesting, open-ended questions seems like a key research skill; call it question taste. Intuitively, a person with question taste is like a child with expert domain knowledge.

Certain groups of people are known for their question taste. For example, consider journalists. They have undergone extensive training in asking questions as part of their university degree. Moreover, they constantly practise the skill of asking questions at work. No wonder they get pretty good at it. Similarly, researchers, psychologists, essayists and investors tend to have good question taste, as questioning is part of their job.

III. #

I assume, perhaps somewhat naïvely, that acquiring question taste is mostly about deliberate practice. The implementational details will depend on the domain you’re working in, though.

A basic strategy might be to identify the people in your area who seem to be asking the right questions and try copying what they do. For example, I’d like to formulate good questions about research agendas in AI safety someday, so I’ll often make note of questions raised by leading researchers in podcasts, blog posts or papers. Sometimes I write down own questions and discuss them with others for feedback, although this is more time-consuming.

Having a safe environment in which to practise asking questions - a training ground - also seems important. If you’re a student, you can attend student-run seminars, where you might feel more comfortable asking questions. Regardless of the kind of questions you aim to ask, you need to find a community to give you feedback. And hopefully, you get some answers too.

As you practise asking questions, you’ll inevitably ask some “bad questions”. Part of the reason why children are so inquisitive is their lack of self-consciousness. They don’t care if their questions make them sound stupid. So, in order to ask good questions, make a conscious effort, find a supportive community and maybe don’t take yourself too seriously.

This is argued in Bryan Caplan’s book The Case Against Education. ↩︎

A taxonomy of examples

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 15 Jun 2025 00:00:00 +0000

I’m mildly obsessed with examples. Whenever I feel confused, it’s often because I don’t know enough examples. Proofs can be confusing too, if some step is poorly explained. But that kind of confusion tends to be local. You can still have a good grasp of the theory. If you don’t know enough examples, you feel generally lost. Textbooks lacking in examples end up being dry. In contrast, books with well-chosen examples are a pleasure to read¹.

According to the Cambridge Dictionary, an example is “something that is typical of the group of things that it is a member of”.

In mathematics, there are many kinds of examples, and they all serve different purposes. Traditionally, fields like topology and measure theory place a greater emphasis on counter examples. In algebra, there are more prototypical examples; definitions are often followed by an exhaustive list of objects of that category². In the context of mathematics, the dictionary definition is too simplistic! So here is an attempt at classifying the main kinds of examples, or, alternatively, an examples appreciation post.

Understanding definitions #

First, there are the examples helping us understand definitions.

Prototypical examples: examples just complex enough to capture the essential properties of an object.
- Fundamental groups: the fundamental group of the circle $\pi_1(\mathbb{S}^1, x_0)$.
- Transcendental field extensions: $\mathbb{Q}(\pi)$.
- Modular forms: the Poincaré series $P_{m, k}(z)$.
Canonical examples: the canonical construction of something - the construction you get which involves the least arbitrariness.
- Stochastic processes: the coordinate process.
- Embeddings into bi-duals: take the embedding $V \to V^{**}$ sending $v$ to the evaluation.
- Rings: the integers $\mathbb{Z}$.
Foundational examples: examples of special interest and which form the basis for further theory.
- Anything analysis: the Gaussian $\phi(x) = e^{-x^2/2}$.
- Stochastic processes: the simple random walk on $\mathbb{Z}$.
- Modular forms: the $j$–invariant.
Basic examples: the simplest possible instance of an object. The kind of example an uncreative student might cite in an exam³.
- Groups: the trivial group $G := \{e\}$.
- Martingales: the martingale $(X_t)_{t \ge 0}$ where $X_t \equiv 1$ for all $t$.
- Banach space: just take $\mathbb{R}$.
Non-examples: objects which are not instances of something.
- A non-ring: the natural numbers $\mathbb{N}$.
- A non-tempered distribution: the function $e^t$.
- A non-Artinian ring: the integers $\mathbb{Z}$.
Computational examples: examples involving a computation and that illustrate how to work with a given object.
- Compute $\mathbb{R}[x] \otimes \mathbb{C}$.
- Find the Euler product expansion for the Dirichlet generating series of the Möbius function $\mu = 1^{-*}$
- Check that $c^{-1/2} W_{ct}$ is again Brownian motion.
Pathological examples: examples illustrating distinctions between different notions, or the limitations of a certain concept.
- Differentiability $\neq$ continuity: consider the Weierstrass function.
- Connectedness $\neq$ path connectedness: because the Topologist’s sine curve.
- Lebesgue integral $\neq$ Riemann integral: try integrating the Dirichlet function, the characteristic function of the rationals.
Toy examples: examples you can manipulate to have some desired properties. When looking for counter examples, you typically start with some toy example.
- Random variable: define $$X := \begin{cases}\alpha, & \text{with probability } p, \\\ \beta, & \text{with probability } 1 - p.\end{cases}$$
- Measure theory: simple functions.
- Galois groups: consider $\mathrm{Gal}(\mathbb{Q}(\sqrt{\alpha}):\mathbb{Q})$.
Real-world examples: a real-world scenario involving the object of interest. These help motivate the study of the given object and are useful for gaining intuition.
- Stochastic processes: a Poisson process as a description of the number of raindrops falling in a given square.
- PDE: the heat equation.
- Graph theory: a network models the flow of some fluid.

Understanding results #

Second, there are the examples helping us understand results. I’ll count applications of theorems among these kinds of examples.

Counter examples: examples illustrating how a given result breaks down if we drop assumptions⁴.
- Fatou’s lemma: we really do need non-negativity. Consider the sequence $f_n := - 1_{[n, n+1]}$.
- Open mapping theorem: surjectivity is necessary, since the zero map isn’t open.
- Nullstellensatz: the field $K$ must be algebraically closed. For $K = \mathbb{R}$, the ideal $I = (x^2 + 1)$ is maximal but not of the form $(x - a)$.
Basic applications: checking that the conditions of a theorem are satisfied, apply the theorem and see what you get. These examples are particularly useful if the theorem statement seems involved.
- Hurewicz’ theorem: use it to find $H_1(\mathbb{S}^n)$.
- Dedekind-Kummer: compute the factorisation of $(p)$ in a ring of integers.
- Dirichlet’s unit theorem: the theorem allows us to verify that $\mathbb{Q}(\sqrt{d})$ has finite unit group.
Non-basic applications: or corollaries. Deduce something interesting from a theorem.
- Optional stopping: allows you to compute laws of hitting times.
- The ring of integers is Dedekind: this gives us a satisfying proof of Fermat’s theorem for primes that are sums of squares.
- Hausdorff-Young: an application of Riesz-Thorin tells us that the Fourier transform defines a bounded linear operator from $L^p$.
Amusing applications: examples mentioned during lectures for amusement.
- Borsuk-Ulam: there are two antipodal points on earth with the same temperature.
- Mean-value theorem: you can catch someone over-speeding using the mean-value theorem.
- Four colour theorem: you can colour a map using just four colours so no two adjacent countries have the same colour.
Satisfying applications: prove a well-known theorem as an application of the fancy theory you developed.
- The insolvability of the quintic: “just” an example application of Galois theory.
- The central limit theorem: falls out of the computation of the characteristic function of the scaled sum.
- The fundamental theorem of algebra: an application of Liouville’s theorem.

Are examples the core of mathematics? The answer is very much a matter of personal taste. I’d say yes. Apparently von Neumann once said

“In mathematics you don’t understand things. You just get used to them.”

There’s some truth to this. And the way we get used to things is by studying examples.

Thanks to Alois Schaffler for suggesting the last kind of example.

Basic Category Theory or The Napkin are amazing in this regard. ↩︎
In Abstract Algebra by Dummit and Foote, some definitions are followed by more than ten (!) examples. ↩︎
I once cited $\mathbb{Q}$ as an example of a non-Banach space in an exam. The graders probably weren’t particularly happy with my response, but I received full points. ↩︎
In my experience, memorising counter examples is the best way to memorise assumptions of a theorem. This is somewhat counter intuitive, given that this involves memorising more information. ↩︎

On the joy of customising

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 08 Jun 2025 00:00:00 +0000

Why do so many people use Safari, the jarring black VSCode colour theme and Overleaf? Because it’s the default option.

Switching to a non-default option requires conscious effort. First, you need to realise that your current setup is suboptimal: the probability of your current setup being the optimal one is infinitesimal. But if you knew about the benefits of making a few small changes, you’d already have made them. So you do need to believe that switching is worthwhile, which, indeed, is a leap of faith.

The pros are all believers. Most software engineers customise their code editors and terminal shells heavily. If you spend all your working hours working on a computer, spending one Sunday improving your working environment is well worth it. Similarly, I know some professors at ETH are particular about their writing tools, only using fountain pens and high-end chalk.

But if you’re neither a software engineer nor an ETH professor, tinkering with command-line interface (CLI) tools and splurging money on fountain pens might feel indulgent. This makes up another psychological barrier towards switching from the default. In fact, you might question whether you’re even entitled to use the tools of the pros. And to some extent, this sentiment is valid.

However, I still think most people err on the side of customising too little, especially when it comes to tech. Most of us are heavy tech users; the average screen time globally seems to be over 6h. When I say someone has a good tech setup, I mean that they use appropriate software for their computer-related tasks efficiently. It is much like having a good office with a comfortable chair and a proper desk. However, it need not require expensive purchases.

Adding a few selected tools to your setup can massively improve productivity. There are some general programs useful to any computer user, like Google Drive, Google Calendar and Raycast. I also think having a dedicated note-taking system, whether it be in Obsidian, Notion or Apple Notes, pays off. However, you’ll get the most leverage from finding the tools relevant to the tasks you’re doing and mastering them.

For example, I mostly use my MacBook for typesetting, writing and coding. For typesetting, I use NeoVim with custom snippets, inspired by Gilles Castel’s iconic VimTeX setup. Nowadays, I also use Typst rather than LaTeX. Although I don’t necessarily type faster in Typst, I spend much less time debugging cryptic error messages. All in all, I estimate that these changes have doubled my writing speed. There are also ergonomic benefits to this setup, as opposed to just writing in Overleaf. As for coding, I’ve found Cursor to be a complete game changer, allowing me to implement experiments in about a tenth of the time. Other than that, I’m “just” using NeoVim keybindings in VSCode and a few CLI tools allowing me to navigate between folders more easily. As can be seen from this post, I have a lot to learn here.

Furthermore, customising your computer makes for a much better user experience. I value good design, so having my preferred colour scheme in Obsidian, VSCode and WezTerm makes my computer a joy to use. I also use the Zen Browser for a cleaner user interface.

Regardless of whether you’re a software aesthete or not, it’s worth exploring tech tools relevant to your work.

Consume less AI safety news

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 01 Jun 2025 00:00:00 +0000

It’s hard staying on top of all the AI safety news. Some people, like Zvi, have basically made this their full-time job.

A common failure mode for forming views on AI safety is consuming too much information. It’s a tendency I’ve observed in myself, as well as in others in the AI safety community.

I think it comes from the urge to solve the AI alignment problem quickly. It’s also an exciting time to be working in AI safety, with many rapid advancements being made. Also, since AI safety is something AI, there’s a lot of general excitement surrounding the area.

So we might imagine someone who starts following all the leading researchers on Twitter, listening to The Cognitive Revolution while commuting and reading LessWrong posts before going to bed. Or maybe they’ll have lengthy discussions about AI governance in WhatsApp groups and watch Robert Miles’ YouTube videos over meals.

But if you’re looking to gain a deeper understanding of the AI safety landscape, this isn’t enough. You’d have to engage more with the material, shifting your creation-to-consumption ratio towards more creation. And you’d have to discuss your views with people in real life and, if possible, engage in a local community¹. A common piece of advice for people aiming to become experts is to focus on building deep models.

Even if your goal is just to get the big picture, the above approach seems needlessly high effort. There are many excellent resources summarising the main ideas in AI safety, such as the the 80,000 hours problem profile. It’s a 60 min read with a very high signal-to-noise ratio.

Either way, binging AI safety-related material to won’t help fix the AI alignment problem. It’s stressful, and perhaps self-defeating. Given that the AI safety landscape is changing so rapidly, much of what we’re seeing is noise. To gain conceptual clarity, perhaps you’ll benefit from consuming less AI-safety related news. Unless you’re working full-time on AI safety, either as a policy-maker or a researcher, this probably won’t negatively influence your ability to do good work.

Hard problem require careful reflection. Although AI advances fast, we must think slowly about how to ensure things go well.

My friend Miles Kodama put it well: “It is easy to BS to a screen”. ↩︎

Understanding the AI alignment problem

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 25 May 2025 00:00:00 +0000

Broadly speaking, the AI alignment problem refers to the problem of ensuring AI systems do what we want them to do. I like the definition used by Anthropic here:

“build safe, reliable, and steerable systems when those systems are starting to become as intelligent and as aware of their surroundings as their designers”

The general idea is pretty simple. But there are many things to unpack here. Understanding what this means in practise is hard. First, why might we end up training unsafe AI systems? Even if you think this is possible, it’s not clear what regulations might be appropriate. Here are some metaphors I found particularly useful for gaining a deeper understanding of some aspects of AI alignment.

The eight-year-old CEO #

In this excellent blog post¹, Ajeya Cotra asks you to imagine the following scenario:

Imagine you are an eight-year-old whose parents left you a $1 trillion company and no trusted adult to serve as your guide to the world. You must hire a smart adult to run your company as CEO, handle your life the way that a parent would (e.g. decide your school, where you’ll live, when you need to go to the dentist), and administer your vast wealth (e.g. decide where you’ll invest your money). You have to hire these grownups based on a work trial or interview you come up with – you don’t get to see any resumes, don’t get to do reference checks, etc. Because you’re so rich, tons of people apply for all sorts of reasons.

Your candidate pool includes:

Saints – people who genuinely just want to help you manage your estate well and look out for your long-term interests.

Sycophants – people who just want to do whatever it takes to make you short-term happy or satisfy the letter of your instructions regardless of long-term consequences.

Schemers – people with their own agendas who want to get access to your company and all its wealth and power so they can use it however they want.

Deciding whom to hire is extremely difficult - you’re just eight! In this analogy, humanity is the eight-year-old CEO. Hiring a candidate is like training the superhuman AI model which will best serve our interests.

Building planes #

Suppose aerospace engineers have developed a new plane model. It’s energy-efficient, cheap to produce and has increased passenger comfort. However, the engineers don’t fully understand the internal workings of the engine. During testing, the engine seems to work alright. The engineers identified a few issues, but these could all be fixed quite easily. Would you be comfortable with this plane being produced for commercial use?

Here, the AI models are like the engines. We know how to build AI models capable of writing poetry and conducting PhD-level research, but our understanding of how these models learn is relatively limited². So, we should probably reflect more carefully on how we’re deploying LLMs.

Drug regulation #

It takes years for newly developed drugs to reach consumers. First, you need preclinical trials. Then you carry out clinical trials in three distinct phases. This done, you need the approval of a regulatory agency, such as the FDA. It’s not uncommon for the entire process to take 10-15 years. Given that we subject drugs to such rigorous testing, why not do the same for LLMs?

I first heard this analogy in this podcast (Swedish, sorry), where Olle Häggström makes the case for AI slowdown. I think the above analogy is quite compelling, although I don’t fully share his views.

The hustler #

Imagine a person trying to learn a new skill, say playing Go. He has memorised all textbooks on Go ever published by heart, as well as all the games played by professional Go players. Moreover, he’s extremely hardworking: he plays roughly 1.5 million games against himself per day. (He doesn’t need any sleep, and he happens to think very quickly.) Given the amount of practise he gets, how can normal humans hope to defeat him?

Here, the hustler is similar to an RL system. To me, this analogy makes the prospect of an intelligence explosion seem much more plausible³ and less sci-fi-ish.

Final thoughts #

Finally, to understand the alignment problem, I think it’s also worth appreciating the potential impact of superhuman AGI. To me it seems like superhuman AI could be about as transformative as the industrial revolution. At the very least, I’d expect it to be as impactful as electricity. So, the ensuring the development of AGI “goes well” seems like a key problem of our time.

This is one of my all-time favourite pieces on AI alignment. ↩︎
I don’t count “just backpropagate” as a satisfactory answer! ↩︎
This analogy was inspired by a conversation with Samuel Ratnam at EAG. ↩︎

A semester paper retrospective

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 18 May 2025 00:00:00 +0000

This is a follow-up on my previous article, where I share some thoughts on the process of writing a semester paper.

Lots of disclaimers: this is all very specific to my experience: the topic of my paper, my supervisor, my workload in others courses, etc. Also, I’m certainly in no position to give advice on mathematical writing; these are just reflections on what worked and what didn’t work for me.

On writing papers #

Here are some principles for writing mathematical papers that I tried to follow, mostly based on feedback from my supervisors. I received plenty of useful feedback, but here are the points I found the most useful:

Top-down exposition: concretely, this meant explaining how all lemmas would come together before proving each lemma. This minimises the amount of task-switching for the reader. In addition, this helps motivate the lemmas and allows the reader to decide which parts to skip.
Short proofs: as a rule of thumb, I tried breaking up proofs in such a way that proofs of subresults would fit on one page. To my surprise, in most cases, this was doable.
Reducing cognitive load: a piece of feedback I received from Vivian was to try reducing the cognitive load of the reader; it’s also a major theme here. This idea really resonated with me. Concretely, this meant doing things like:
- Restating parameters: for example, one might go “Recalling that $X = \text{definition of }X$, we obtain…” rather than “By our choice of $X$, ….”.
- Creating indices of notation and parameters: very relevant if your proof is notation-heavy.
- More descriptive text: rather than writing “we have” or “thus” before a computation, I tried describing which device I was using. Mentioning that you’re using, say, a union bound doesn’t take up much additional space on the page, and it makes for a better reading experience. (Also, if you’re doing analytic number theory, you need more alternatives to “we have”!)
- Indicating what won’t be proved: in order to keep the report at a reasonable length, I had to omit the proofs of some preliminary results and avoid repeating similar arguments. I tried making this clear to the reader, so I wouldn’t leave them hanging.
What I would have found useful: initially, I didn’t know how many details to include. If I’d try explaining all details I’d struggled with when reading the proof, the text would be too verbose! Vivian had some good advice here: aim for the level of detail you’d liked when reading it for the first time.

For future papers, I’m considering trying the following:

Including dependency graphs: although a bit unconventional, these can be tremendously helpful for long and convoluted proofs. See e.g. my piece on dependencies.
Commenting on dead ends: a few words about why the naïve approach fails can be very illuminating. This can help motivate the use of some very complicated tool and potentially save the reader a lot of time. While remarks on failed proof attempts can sometimes be found in the “Discussion” section of a paper, I think like they aren’t given as much attention as they deserve (an unfortunate instance of publication bias).
Reducing cognitive load, even more: if I’d kept this in mind during the entire writing process, I think I’d done a few things differently. For further ideas on how to reduce cognitive load, I highly recommend the previously mentioned piece.

Lessons #

If I were to write a thesis or paper again, I would have done a lot differently. Here are the main changes:

Do it in one go: it would have been much more enjoyable and efficient doing most of the semester paper within a much shorter period of time. If you’re trying to understand an involved argument, you need all the relevant notions floating around in your brain at the same time. Don’t read one lemma per week.
Request additional feedback: if your supervisor is kind to offer additional feedback, that’s extremely valuable.
As soon as you get it, typeset: you’ll have to do it anyway, and you might as well do it when you understand it. This also helps corroborate your understanding.
Know when to ask for help: I wasn’t sure how long it was reasonable for me to be stuck on a particular passage before asking for help, and I know supervisors have different preferences here. Trivial fix: ask your supervisor “For how long should I be stuck before asking for help?”
Knowing how to ask for help: at first, I’d ask questions during meetings. But I soon began emailing a list of questions to my supervisor a few days before our check-ins. That way, I think we both got more out of the meetings.
Typesetting takes time: it seems to be a law of nature that TeX:ing always takes longer than expected. This happened as I was writing my bachelor thesis too. Despite having a pretty sophisticated NeoVim LaTeX setup with snippets, typesetting took twice as long as I expected. For future papers, I’ll probably go with Typst.

Two results from probabilistic number theory

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 11 May 2025 00:00:00 +0000

I recently wrote a semester paper on probabilistic number theory. I’m very grateful to both of my supervisors, Dr. Vivian Kuperberg and Prof. Dr. Emmanuel Kowalski, for their insights and suggestions. Here’s the abstract:

This report is an exposition of two central limit theorems in probabilistic number theory. We begin by introducing preliminary results from number theory and probability theory. Then we prove the Erdős-Kac theorem for the asymptotic behaviour of the prime divisor counting function. The majority of the report is devoted to Radziwiłł and Soundarajan’s recent proof of the Selberg central limit theorem for $\log |\zeta(\frac{1}{2} + it)|$.

You’re most welcome to have a look at the actual semester paper. Here I’d like to give a more informal discussion of its contents.

“Deterministic” number theory #

A central theme in analytic number theory is deriving estimates for averages over arithmetic functions, e.g. something like

$$\sum_{n \le x} f(n) = \text{main term} + \text{error term},$$

where $f: \mathbb{N} \to \mathbb{C}$ is some arithmetic function. Many “named theorems” are estimates of the above form. For example:

If we let $f(n) = x^{-1} 1_P(n)$, where $1_P$ is the indicator of the set of prime numbers, then we get Mertens’ second theorem.
If instead $f(n) = d(n)$, the function counting the number of divisors of a positive integer $n$, then a theorem of Dirichlet asserts that

$$\sum_{n \le x} f(n) = x \log x + (2 \gamma - 1)x + O(\sqrt{x}).$$

Actually, the Prime number theorem is equivalent to an estimate of the above form. Just take $f$ to be the von Mangoldt function $\Lambda$, defined by

$$\Lambda(n) := \begin{cases}\log p, & n = p^k \ \text{for some prime } p, \\\\ 0, & \text{otherwise}.\end{cases}$$

Probabilistic number theory #

In my semester paper, I focused on two central limit theorems from probabilistic number theory: the Erdős–Kac and Selberg central limit theorems. Here, the approach is slightly different.

Sums and integrals as expectations #

First, we let $f: \Omega \to \mathbb{C}$ be an arithmetically defined quantity more generally; typically $\Omega = \mathbb{N}$ or $\mathbb{R}$. So we could take $f$ to be an arithmetic function, but now we also allow for functions of arithmetic functions. For example, if we take $g: \mathbb{N} \to \mathbb{C}$ to be an arithmetic function and post-compose with $x \mapsto x^2 / \sqrt{\log \log x}$, then we could have $f: \mathbb{N} \to \mathbb{C}$ be a weird-looking expression defined by

$$f(n) := \frac{g(n)^2}{\sqrt{\log \log n}}.$$

To highlight the connection with probability theory, fix a positive integer $x$ and let $U_x$ be a random variable uniformly distributed on $\\{1, ..., x\\}$. Then we obtain

$$\frac{1}{x} \sum_{n \le x} f(n) = \mathbb{E}(f(U_x)).$$

So, estimating $x^{-1} \sum_{n \le x} f(n)$ is the same as estimating an expectation.

Similarly, the problem of estimating the integral

$$\frac{1}{x} \int_0^x f(y) \ dy$$

also comes down to estimating an expectation. In the continuous case, let $x$ be an arbitrary positive real number and take $U_x$ to be a random variable uniformly distributed over $[0, x]$. Then the above integral is precisely $\mathbb{E}(f(U_x))$.

Natural questions #

This naturally prompts the following questions:

Could we also estimate the variance of $f(U_x)$?
Could we even say something about the asymptotic distribution of $f(U_x)$ as $x \to \infty$?

Ideally, we can prove something like this:

“Let $U_x: \Omega \to \Omega_x$ denote a random variable uniformly distributed on $\Omega_x := \\{1, ..., x\\}$ (or $[0, x]$). Let $Y$ be a random variable with some simple distribution. Then as $x \to \infty$, we have

$$f(U_x) \xrightharpoonup{} Y,$$

where $\xrightharpoonup{}$ denotes convergence in distribution.”

A remarkable fact is that some rather convoluted expressions involving arithmetic functions can converge in distribution to standard normal random variables as $x \to \infty$.

General proof strategy #

Proving results of this form involves machinery from both number theory and probability. We typically use number theory to massage the expression $f(U_x)$, so it can be approximated by some simpler non-arithmetic random variable. This done, we can use limit theorems from probability to finish off the proof.

The Erdős-Kac theorem #

This serves as a good “toy example” for the proof strategy outlined above. Also, it’s not too difficult deriving a heuristic proof; for details, see page 14.

First, some notation. Let $\omega$ denote the function indicating the number of distinct prime divisors of a positive integer. For example, $\omega(6) = 2$ since $6 = 2 \cdot 3$, while $\omega(9) = 1$, since $9 = 3^2$. Then we have:

Theorem. (Erdős-Kac, 1940). Let $U_n$ be a random variable uniformly distributed on $\{1, ..., n\}$ and define

$$W_n := \frac{\omega(U_n) - \log \log n}{\sqrt{\log \log n}}.$$

Then the sequence $(W_n)_{n \ge 1}$ of random variables converges in distribution to a standard normal random variable as $n \to \infty$.

Selberg’s central limit theorem for $\log |\zeta(1/2 + it)|$ #

Let $\zeta$ denote the Riemann zeta function, defined by

$$\zeta(n) := \sum_{n \ge 1} \frac{1}{n^s}.$$

The Riemann hypothesis, perhaps the most famous open problem in mathematics, says that the only zeros of the zeta function are of the form $s = -2k$ for positive integers $k$ or lie on the critical strip $\\{1/2 + it : t \in \mathbb{R}\\}$.

Now fix a positive number $T > 0$, and consider the random variable

$$L_T := \log |\zeta(1/2 + iU_t)|,$$

where $U_t$ is a random variable uniformly distributed on the interval $[-T, T]$. Considering just how hard the Riemann hypothesis is, it’s pretty surprising that the behaviour of $L_T$ is relatively well-understood. In fact, we have:

Theorem. (Selberg, 1946). Let $L_T$ be the random variable defined as above, and set

$$M_T := \frac{L_T}{\sqrt{\frac{1}{2} \log \log T}}.$$

Then the sequence $(M_T)_{T \ge 0}$ of random variables converges in distribution to a standard normal random variable as $T \to \infty$.

In my semester paper, I cover the 2016 proof due to Radziwiłł and Soundarajan. Just as in the Erdős-Kac theorem, one can get an intuitive sense of why this result is plausible.

Final words #

For details, I encourage you to have a look at the actual paper. I hope you enjoy reading it as much as I enjoyed writing it.

On decision fatigue

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 04 May 2025 00:00:00 +0000

I. #

Life involves plenty of hard decisions. Should I switch jobs? Should I marry him? Should I move abroad? Then there are the “easy” decisions: which pyjama to wear or which brand of laundry detergent to buy. Although it doesn’t matter which option you choose here, choosing is hard. In fact, if you think of buying laundry detergent as an optimisation problem, taking into account things like social impact, price and quality, I’d expect the optimisation problem to be NP-hard! Here’s Zvi:

When you have a choice, you must stop what you’re doing, and choose. […] Making a random slash arbitrary choice might not be hard (although sometimes it is) but first you have to choose to choose at random. If you choose not to decide, you still have made a choice. There’s no escape!

This leads to one of the big curses of adulthood - decision fatigue.

II. #

Most of us are aware that choosing is draining. Steve Jobs wore black turtlenecks to reduce the number of decisions in a given day. Should I Stay or Should I Go is basically a three-minute rant about how terrible it is making decisions¹. The reason going to IKEA can be so tiring is that it involves making many small choices. Getting to choose among many options is only fun up to a certain point.

According to Wikipedia, decision fatigue is “the deteriorating quality of decisions made by an individual after a long session of decision making”. Decision fatigue arises from a broader phenomenon called ego depletion. Intuitively, this is the idea that willpower works like a battery. If we exert a lot of mental effort at one task, then we cannot exert as much mental effort at subsequent tasks without recharging. Making decisions drains our willpower battery.

The phenomena of decision fatigue and ego depletion have been demonstrated in a number of amusing studies. A striking example is the study on Israeli judges, demonstrating the so-called hungry judge effect.

Computer science offers another way to think about decision fatigue. While a psychologist might speak of ego depletion, a computer scientist might speak of computational complexity. Here’s from Algorithms to Live By by Brian Christian and Tom Griffiths:

“One of the implicit principles of computer science, as odd as it may sound, is that computation is bad: the underlying directive of any good algorithm is to minimize the labor of thought. When we interact with other people, we present them with computational problems—not just explicit requests and demands, but implicit challenges such as interpreting our intentions, our beliefs, and our preferences. It stands to reason, therefore, that a computational understanding of such problems casts light on the nature of human interaction.”

Further, the authors go on to advocate for what they call “computational kindness”. When interacting with people, we should try sharing the computational cost of making decisions. If you prefer a particular restaurant, don’t hesitate to indicate your preference. Although you’re trying to be polite by saying you’re indifferent, the other person might be relieved to hear you’re craving a particular kind of food.

III. #

The first step in dealing with decision fatigue is to notice it (as with so much else related to mental well being, I guess). Then one is in a better position to either make a quick, haphazard decision or recharging one’s willpower battery, so one can make a better decision later. This is easier said than done, though.

For example, I recently planned a trip to London, booking flights, finding accommodation, coordinating with others, etc. Because I was excited about the trip, I didn’t notice when decision fatigue kicked in. If I’d known that it was decision fatigue, I wouldn’t have planned the trip in as much detail.

In practise, noticing decision fatigue is hard. One can always practise the skill of noticing, although it takes a lot of time. Another strategy would be trying to identify situations involving many choices and recalling how it felt. For instance, typical triggers of decision fatigue might be shopping, planning or coordinating with others. Analysis paralysis or a feeling of frustration might be good proxies for decision fatigue.

But one can also take preventative measures. We can cut out a surprising number of choices from our everyday lives by coming up with default actions for various situations. For example, I don’t want to only wear black turtlenecks, but I do have a default outfit - a uniform of sorts. In Google Calendar, I also have a calendar describing my ideal week, so I don’t have to decide which lectures to attend on a day-to-day basis. Or at the supermarket: pick the cheapest toothpaste.

Insignificant non-recurring choices can be batched, so one doesn’t go into choosing mode too often. One can even gamify the experience of making choices. For instance, I could have challenged myself to plan the trip within a given time frame.

IV. #

One of the best things about being an adult is that you get to do whatever you want. Getting to choose is a privilege, after all. It’s a privilege we should use.

If you speak Swedish, I recommend listening to Amanda Ginsburg’s I de många valens land. ↩︎

On good conversations

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 27 Apr 2025 00:00:00 +0000

I. #

Some studies suggest we spend 80-90% of our waking hours talking with others. For a normal person, that’s at least 12 hours (!) per day. We spend most of our working hours talking to others, whether it be in meetings, lectures or exercise classes. When we’re not working, friends, roommates or partners might be around. The hours add up.

Why do we spend this much time talking with others? Above all, interesting conversations are among the greatest pleasures in life. There’s nothing like a good conversation over a good meal. But talking with others can also be very productive. For example, talking with people smarter than you is a phenomenal way of learning. It’s like you’re downloading part of someone else’s worldview into your own head. Of course, there are other ways of achieving the same thing. However, in my opinion, one-on-one discussions come the closest to a loss-free download.

For this reason, it’s natural to ask how we can have better conversations. Similar questions are often raised in the context of dating (“What are good topics to talk about on a date?”) or business (“How can we have more productive meetings?”). However, this question seems relevant to everyone.

II. #

Several classic Russian novels go something like this: some characters are sitting on a train, they begin chatting and one thing leads to the next. That leads to an 800-page novel. The conversations they have are fascinating, although a bit exaggerated.

I often have good conversations when I’m traveling. It’s as if me and my travel companions have this tacit agreement: “Okay, we’re all somewhat bored. We can’t do small talk for another five hours, so let’s talk. For real.” For this reason, people often end up being more open with one another.

Moreover, simply being in a different environment can help. If you and a colleague are on a business trip, you’re less likely to ask the usual questions about the weather or life back home.

Takeaway: the norms of the conversation matters a lot. If everyone implicitly agrees on the purpose of the discussion, things become a lot easier. Sometimes, there’s no need to say it out loud. But it can definitely help. For example, it might be better starting the phone call by saying “Hi, I need advice about you-know-who.” Or you might agree on a set of norms for the conversation. For instance, I recently learned that some people in rationalist circles will put a five minute timer whenever someone brings up philosophy of mind, a topic which often leads nowhere.

III. #

Many scientists, artists and writers would have something like an intellectual spouse - a close collaborator with whom they’d discuss ideas. Famous examples include Watson and Crick, Lennon and McCartney or Sartre and de Beauvoir.

Discussing is a skill. Doing it with the same people, at the same time and at the same place is a bit like practising regularly. Once you know each other sufficiently well, there’s no need to agree on conventions again. You just pick up from where you left off.

IV. #

The Arabic proverb goes “Speech is silver, silence is golden.” But in order for there to be silence, you need to be in a space with little background noise. Here comes an obvious point: having good discussions in noisy environments is much harder. I feel like this point tends to get neglected in practise, so I find it worth emphasising. Most restaurants, cafés or co-working spaces can be quite rowdy. Instead, consider going for a walk. Apparently Steve Jobs loved going for walking meetings, perhaps for this reason.

V. #

From an information-theoretic perspective, good conversations are nothing short of miracles. You’ll have several people communicating complex ideas at an astonishing speed. There are no mind-reading devices involved, just words. In my own experience, having a good conversation largely comes down to synchronisation.

As a quick side note, there’s a reason I’m using the word “synchronisation” rather than “communication”. “Communication” can also be interpreted as a one-sided process, where one party shares information with another group. But in a good discussion, information usually flows both ways.

One way of synchronising better is asking more questions. Not only do questions help eliminating confusion, but they also make the conversation much more enjoyable. Depending on the situation, asking questions might also be necessary for everyone to feel included. Moreover, as pointed out by Ben Kuhn here, asking questions puts you in a better position to help whoever you’re talking to. Without any background information, you’re likely to give bad advice. And you can only acquire background information by asking questions.

Another useful technique is paraphrasing. The goal here isn’t to sound smart, but to synchronise with your conversation partner. In practise, this means I’ll try rephrasing the other person’s idea and asking if it’s accurate.

VI. #

It’s not just about being in sync, though. A good conversation also needs to be heading in an interesting direction. If you all know you’re heading towards a dead end or that you’re going in circles, then it might be worth pointing this out (kindly, of course). Chances are people will agree and be glad someone noticed. It’s a bit like when the host declares the party is over at 2 am, and you’re grateful for someone forcing you to go home and get some sleep. Another benefit of moderating the discussion is that you simultaneously encourage the group to metacogitate. Similarly, if you feel uncomfortable with a particular subject, it might be a good idea to say you’d prefer talk about something else, rather than being passive aggressive.

VII. #

How many of our 12 hours of conversation are of high quality? Quite few, I suspect. And this completely OK. Having good conversations can be draining too. It’s a form of flow. Even Mihaly Csikszentmihalyi, the psychologist who coined the term “flow”, remarked that we cannot spend all our lives flowing. So there’s no need to cut all small talk.

But some conversations matter more than others. In fact, certain conversations can be life-changing. They make life more enjoyable, generate creative and scientific insights and connect us to others. What if we could improve the quality of these conversations ever so slightly?

Building time machines

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 20 Apr 2025 00:00:00 +0000

There’s a folder on my computer called “Time machine”, where I collect files on today’s me. It’s not exactly an FBI record on myself, but rather like a digital self portrait. After doing this over a longer period of time, I’ll share the folder with future versions of people I care about.

I. #

I like poring over old photos. I think it’s something I’ve picked up from my mom. She’s something of a camera terrorist – she’ll always pull up the camera and have everyone else pose. As a teenager, I used to find this pretty embarrassing. But now I’m glad she ignored my complaints.

A key theme in the book Thinking, Fast and Slow is the distinction between the experiencing and remembering self. The experiencing self is the part feeling pleasure and pain, while the remembering self reflects on past experiences. While we’re often told to carpe diem more, we’re rarely told to try pleasing our remembering selves. But looking back on memories can be a huge source of joy.

In A little book about happiness, Michael Dahlén, professor of well-being at the Stockholm School of Economics, describes a fascinating experiment in which participants were asked to estimate their well-being at three points in time: past, present and future. On average, the graph looked like a slanted “V”, the lowest point being the present. Perhaps looking back can make you more appreciative of what you have.

If that all sounds a bit woo woo, that’s OK. If you want, you can also use your time machine to spot cognitive biases. For instance, I have a list of things I’ve changed my mind about. As highlighted in Stumbling on Happiness (bad title, good book), we consistently fail to appreciate just how much we change. I also have a document with my worries. When revisiting past entries, I notice a tendency of taking things too seriously.

The end product, your time machine, is something you can share with others. In due time, I’ll share mine with the people who matter the most to me. For instance, I’d like to share my time machine with family members, long-time friends and my future partner. And in the very distant future, I might show it to my teenage kids, allowing them to get to know their mother better.

II. #

There’s a lot of latitude in how to build a time machine. My journal makes up a big portion of my time machine. But there are also other files in there. One of the benefits of the digital age is that we can store huge amounts of data very easily, so you may as well be experimental.

In secondary school, a friend of mine started a trend of creating monthly playlists on Spotify¹. I was quick in hopping on the trend, and I’m very grateful I did. Today I have a complete record of how my music taste has evolved since August 2015². But I’m far from the only one doing this kind of thing. For example, Luke Muehlhauser publishes a list of music recommendations on a quarterly basis on his blog.

I’ve also kept a reading diary since 2016. It must have started out as a vanity project. Having read a boring classic, I assume I wanted to remember that I’d finished it. It’s amusing seeing how my preferences have changed over time. My reading log is just a long markdown document on my computer³, but you could also use a platform like StoryGraph. As of a couple of months ago, I’ve also been saving blog posts that particularly resonated with me. However, you could go beyond music and reading lists. Luke Muehlhauser also compiles lists with video games. If you’re a big podcast listener, take note of which podcasts you particularly enjoyed in a given month.

This is all a lot of work, especially if you consume a lot of content. Be selective, but be consistent. By tracking your media diet over the course of years, you’ll learn how you’ve changed and gain more self-knowledge. You are what you eat, right?

Other things could go into your time machine too. You can get to know someone else in a surprisingly short amount of time by asking the right questions. In fact, there are various compilations of questions meant to foster closer relationships, the most famous one being the 36 questions to fall in love. You could grab your favourite questions from that list and answer them in writing every year, say.

I have two-three documents of this genre, but I’m planning on adding more. As mentioned above, there’s the list of things I was wrong about, as well as the list of things preoccupying me. Following the example of Nabeel Qureshi, I also have a “principles” document.

III. #

Humans have always been building their own time machines in the form of literature and art. But a time machine can be just about anything. Building one doesn’t require artistic talent, just consistency. Nowadays, you can build a digital time machine with minimal effort. Since files don’t take up any physical space, there’s no need to clutter your house with keepsakes.

I was recently given a necklace which belonged to my great grandmother. It’s a beautiful piece of jewellery. But it doesn’t tell me who Ellen was. I hope I’ll be able to gift future generations something more.

Thank you Nina Ivarsson. ↩︎
The playlists from 2015-2017 are on my parents’ account, but the more recent playlists can be found here. ↩︎
For excerpts of my reading log, see the library. ↩︎

How I use LLMs

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 13 Apr 2025 00:00:00 +0000

About 40% of students in the ETH main library always have a Chat-GPT tab open. I soon decided to try using LLMs for my own studies (because the wisdom of the crowd is a real thing). I haven’t figured out how to best use LLMs for my coursework, but I’m experimenting with various approaches.

Getting unstuck #

For me, a big time sink is getting stuck on details. I usually go over the lecture notes after lectures, trying to work out the steps I didn’t follow with pen and paper. Ideally, I’d do this sitting next to a friend - it’s very convenient having someone whom to ask nearby. As Nate Soares put it here:

“The problem is, most of the time that I get stuck, I get stuck on something incredibly stupid. I’ve either misread something somewhere or misremembered a concept from earlier in the book. Usually, someone looking over my shoulder could correct me in ten seconds with three words. ‘Dude. Disjunction. Disjunction.’”

But studying with a friend isn’t always possible. If there’s a point of confusion I cannot resolve myself after making a reasonable effort, asking an LLM might help. Formulating a good question is always an instructive exercise. Moreover, nine times out of ten, the response is useful. Even if the LLM doesn’t entirely solve my problem, it might reference relevant concepts or serve as a sanity check. Sometimes I’ll learn that my approach was completely mistaken - and that’s certainly useful too!

Hints #

In the classic How to Solve It, George Pólya famously noted that mathematics isn’t a spectator sport. By generating hints, LLMs can be an aid in the problem-solving process too. Just have an honest attempt at the problem before consulting an LLM, and tell the LLM to not give away the entire solution. But it’s important to notice when one is stuck and ask for help. For someone like me who usually waits too long before taking hints, the ease of generating hints with Chat-GPT makes a huge difference.

The big picture #

LLMs are terrific at explaining high-level ideas. I’m a big fan of learning concepts “top-down”, starting with the big picture before getting into the details. While having more context doesn’t necessarily mean the material sticks better, I find this approach much more enjoyable. I usually ask Chat-GPT to give me the key idea before I look into the details. Apart from this, I regularly prompt Chat-GPT to give me the intuition for something or to motivate concepts. If a lecturer is pressed on time, they’ll cut the motivation bit, rather than leaving out a definition or theorem statement. For this reason, an AI-generated introduction can complement the lectured material.

Some of my favourite prompts include:

“Why do we care about X?”
“What is the main idea behind the proof of X?”
“What’s the intuition for this definition?”

I also find it helpful trying to explain a concept in my own words and asking Chat-GPT to elaborate or check if my explanation is accurate.

Caveats #

All this said, I’d like to add a few caveats.

A friend or teaching assistant could help with the above tasks better than today’s LLMs. They know about your mathematical background and what conventions you’re using. When I interact with chatbots, explaining conventions and providing context adds a lot of overhead. However, this problem seems fixable. Many AI labs are already working on ways to provide more personalised responses by having the chatbot remember information across chat sessions. Students could e.g. upload lecture notes and indicate which parts they’d covered.

AI systems also make mistakes. But this isn’t that big of an issue. Most mistakes are easy to spot, especially if you ask the AI to explain steps that seem fishy. If you point out what went wrong, it will modify the argument. With human guidance, AI systems can get quite far. Also, LLMs don’t need to get all the details right in order to be useful. As Terry Tao noted in this blog post:

“Strangely, even nonsensical LLM-generated math often references relevant concepts. With effort, human experts can modify ideas that do not work as presented into a correct and original argument. The 2023-level AI can already generate suggestive hints and promising leads to a working mathematician and participate actively in the decision-making process.”

Of course, if you want to be dead certain the AI-generated argument is correct, have the AI output a formal proof in Lean.

Another fear of mine, perhaps ungrounded, is basically that LLMs will make us lazy. Learning requires a certain amount of effort, while writing a good LLM prompt is relatively easy. If we use LLMs more and more, will we remove the friction needed for learning? I don’t know whether this fear is valid or if it’s just an instance of “tech panic”. But as long as we set boundaries for our LLM usage, we need not spoil the learning experience.

Where does this leave us? #

It seems, then, as if we could overcome the problems I’ve encountered when tinkering with AIs. My experience with using LLMs as part of my studies has been positive, so I’ll continue exploring ways in which AI can assist. The one thing that LLMs can’t provide, however, is the social aspect of doing maths. Solving problems with others is infinitely more fun than coming up with LLM prompts. If anything, I think the above use cases highlight the importance of doing maths together with others.

Will AI replace mathematicians?

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 06 Apr 2025 00:00:00 +0000

I used to think of maths as the one thing LLMs couldn’t do well. While GPT 3.0 would excel in language-based tasks, it would struggle to solve elementary maths problems. But a lot has happened since then. Over the last year, I’ve come to take the idea of using AI as an aid for doing maths more seriously. In fact, I now believe LLMs might prove new theorems with no human guidance within just 3 years.

First, there was the silver medal at International Mathematics Olympiad (IMO). Although AlphaProof and AlphaGeometry 2 took well over 9h, the time given contestants are given, it’s quite a feat: IMO problems require an element of creativity. Not only that - it was able to formalise its solution in Lean. Lean is still a fairly new programming language, and there isn’t nearly as much training data as for other programming languages. The work of the DeepMind team shows two things: firstly, lots of clever people are trying to build AI systems for doing maths; secondly, apparently their current approach works pretty well. However, as of April 2025, you don’t need an AI specifically trained to do maths in order to solve tricky problems: the new o3 and o4 mini models achieve impressive performance in the American Invitational Mathematics Examination (AIME). What if we use RL to build AI systems specialised in more advanced topics? Perhaps these AIs might prove new theorems. Even if they don’t, they might provide researchers with insights.

Next, several top-gun mathematicians think AI might transform maths research in the next decade. Most notably, Terence Tao has highlighted ways in which machines can help human mathematicians. Here’s from a blog post of Tao:

“I could feed GPT-4 the first few PDF pages of a recent math preprint and get it to generate a half-dozen intelligent questions that an expert attending a talk on the preprint could ask. I plan to use variants of such prompts to prepare my future presentations or to begin reading a technically complex paper.”

In this report from Epoch AI, Richard Borcherds seems equally optimistic about the possibilities of using, predicting that AI might even surpass human mathematicians within 10 years. Overall, I think we’re starting to see a cultural shift in the maths community. People are recognising that AI is a huge deal.

Finally, I observed that 40% of students in the library seem to have a Chat-GPT tab open at all times. These are students doing STEM subjects, such as maths and physics. This seems like an important data point (and this isn’t just because I’m giving more weight to first-hand experience). LLMs are transforming the way students learn, and these are the people who will go on to do research in a couple of years. Chances are we won’t stop using LLMs just because the material becomes more niche. Even if you receive a hallucinatory answer, the LLM might reference a relevant concept, helping you get unstuck. I’m using Chat-GPT for my own studies, and I’m impressed by its explaining abilities. Basically, it can easily handle any concept you’ll come across in a master degree in mathematics. I’ve also prompted Chat-GPT to distill the key ideas from more recent papers and found its responses very helpful.

All in all, I’ve come to shorten my AI timelines quite a bit. But rather than thinking “Will I ever find a job?”, I find myself thinking “What a time to be alive!”

"Just switch it off"

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 30 Mar 2025 00:00:00 +0000

If we develop a rogue AI, couldn’t we just switch it off? This is the obvious objection to the idea that AI could be dangerous. Here “switching off” would mean deleting a model’s weights, so it can’t be deployed. Deleting files is easy enough, so what might prevent us from switching off a misaligned AI?

First, the users need to realise that the model is dangerous. This can be challenging, especially for more advanced models. The key premise here is that AIs will try preserving themselves. That is, they don’t “want” to be turned off - this would prevent them from pursuing their goals. If an AI knows that it will be turned off if it’s misaligned, it might try appearing safe during training. This is commonly known as “alignment faking”. Although this sounds a bit far-fetched, this phenomenon has been observed in some models.

Second, and perhaps more worryingly, is the question of whether we as a society want to “press delete”. Turning off sandboxed AI - AI developed in a secure training environment - isn’t a big deal. The negative consequences, if any, are limited to the few people who can access the model. But a leading AI company has strong incentives against withdrawing potentially unsafe models from the market. Doing this would mean less profit, bad PR and giving away market share to competitors. Besides these economical considerations, there’s the geopolitical aspect. As highlighted in the AI 2027 report, the fear of falling behind in the AI arms race might lead us to deploy even misaligned AI. Even if the people behind the AI wanted to switch off their models because of safety considerations, what would the general public think? Most people would probably be reluctant to stop their favourite LLMs, despite poor performance on safety benchmarks.

Switching off an AI isn’t just a matter of deleting files. It requires us to detect unsafe behaviour, a task that’s likely to become more difficult with more capable models. Then there’s the human factor. Asking that AI companies delete models showing signs of misalignment is asking for a lot. In the future, turning off an AI in a broader sense would require turning off parts of our society.

From a YouTube alumna

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 23 Mar 2025 00:00:00 +0000

Random people on the Internet have played a huge role in my education. I’m not just referring to my coursework at university, but also to “Bildung” more generally. I’ve learned a ton by browsing StackOverflow threads and reading Medium articles. However, I’ve probably learned the most from watching YouTube.

Learning by watching #

Above all, there’s some really high-quality content out there. Nowadays there are full-time YouTubers, working on creating professional, meticulously edited videos. And some channels, such as Kurzgesagt, are even ran by entire teams of illustrators and script-writers. Moreover, because anyone can record themselves and upload it to YouTube, we have world-class experts sharing their knowledge in YouTube lectures¹. This means there are some truly remarkable YouTube videos. For example, here are comments from some 3Blue1Brown videos:

“I dropped out in 10th grade 25 years ago and your videos have inspired me to go back to school.”

“You sir truly deserve an honorary doctorate - just for this video. Your impact to generations of confused engineering and math students will forever ripple through our society.”

“I have a master’s degree in mechanical engineering and I’m starting to think I should redo my whole education from ground up searching for this kind of intuitive knowledge. It’s absurd that I find out explanations which are as intuitive as this one so late in my life. I’m blown away completely! I mean how many bits of information have we stumbled upon during our formal education failing to see how they elegantly relate to each other and form a bigger picture…oh my!”

Moreover, videos tend to be more attention-grabbing than articles. Although most people are unable to read while cooking or brushing their teeth, they can watch videos. So getting started learning has never required less willpower: just search “Introduction to…” on YouTube.

Learning to watch #

Of course, YouTube isn’t designed to be a learning platform. But there ways of optimising for a better learning experience.

The first step is to recognise that YouTube tries maximising user retention. This is a feature, not a bug. It means we can design our YouTube interface such that we end up binge watching informative videos about topics we care about. Here are some ways of achieving this:

Train your algorithm: in your YouTube feed, take 2 seconds to press “Not interested” whenever something irrelevant pops up. It pays off - I find the YouTube algorithm to be surprisingly sensitive to my feedback.²
Keep separate accounts: on a similar note, I have two Google accounts: my main account, and my secondary account. I’ll use my main account for watching “useful” content, while I’ll log onto my secondary account for, well, everything else.
Block channels: use ColdTurkey to block certain YouTube channels.

The second step is to recognise the limits of just watching videos. There’s a reason we don’t abandon more traditional media altogether. When reading a book, I find it much easier recognising when I’m confused. However, after finishing a video, I sometimes find myself completely lost and unable to tell where I stopped following. Moreover, whenever I have a physical textbook, I’ll often refer back to chapters I’ve finished, just to refresh my memory. Here are two partial fixes:

Media notes plugin: the Media Notes plugin for Obsidian is a real game-changer. It allows you to watch YouTube videos from inside Obsidian and take notes with timestamps. When doing this, I seem to engage more with the material. Taking physical notes while watching YouTube is a bit overkill, so this seems like a good compromise.
Rewatch your favourite videos: it’s easy ending up only consuming new content, just because the YouTube landing page is filled with new videos. But it’s worth saving your favourite videos to playlists and rewatching them later.

As I discussed above, there are some hacks for a better learning experience. However, YouTube could also design their platform differently. They could e.g. develop distraction-free mode, enabling the user to remove shorts, ads or sponsored content. From a technical perspective, it’s doable. But we could also go beyond ordinary videos. For example, Andy Matuschak and Michael Nielsen have explored ways of incorporating an element of spaced repetition in videos, making for a more interactive learning experience; see here. In general, I’m excited about integrating modern technology with education.

All this said, I’d like to thank the strangers who have played - and continue to play - an important role in my education.

See e.g. Andrej Karpathy’s channel. ↩︎
My friend Åke Lindblom first told me about this. He apparently has an insanely good algorithm. ↩︎

Look for opinions

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 16 Mar 2025 00:00:00 +0000

Opinionated people can be really annoying. Wherever they go, they try convincing you of their ideas. If you have an opinionated uncle, the Christmas dinner might be ruined by a bitter argument. I’ve certainly had bad experiences with a dinner-table conversations turning into feuds. For this reason, I used to try having fewer opinions. I somehow assumed this meant being more open-minded and mature. Well, no.

Welcoming opinions #

Actually, there are plenty of benefits of actively trying to form more opinions. Even about topics you don’t know particularly well. If you learn with a view towards arguing, then you’ll pay closer attention to the material. I think this has to do with anchoring. If you pick a stance, even at random, you’ll be more emotionally invested. Holden Karnofsky summarised it neatly in Learning by Writing:

By doing this [trying to have a hypothesis and rearticulating it whenever it changes], I try to continually focus my reading on the goal of forming a bottom-line view, rather than just “gathering information.” I think this makes my investigations more focused and directed, and the results easier to retain. I consider this approach to be probably the single biggest difference-maker between “reading a ton about lots of things, but retaining little” and “efficiently developing a set of views on key topics and retaining the reasoning behind them.”

Moreover, chatting with people with strong opinions can also be fun. Say you’re at a cocktail party. Small talk can be quite tiring, at least after a couple of hours. In this situation, I’ll gladly talk to people trying to persuade me of their ideas. Or say you’re hosting friends for dinner, and a friend explains her take on a topic you all like.

Back in school, we were encouraged to form more opinions. Teachers made us write argumentative essays about topics we hardly knew anything about. They know that most 14 year-olds don’t care the slightest about whether fathers should be given two additional weeks of paternity leave, or if the capital income tax should be raised by 1%. But I don’t think it was only meant as an exercise in communicating effectively. It felt as if teachers were saying “Go out there in the big wild world, and look for opinions!”

Chasing opinions #

So why doesn’t everyone have, like, a lot opinions?

I think many people, whether they recognise it or not, resort to some kind of agnosticism for fear of being wrong. However, recognising you’re wrong just means you’re updating your beliefs. It’s not that big of a deal. Also, many people think they aren’t entitled to hold an opinion since they aren’t “qualified”. This is true for areas in which there’s a clear distinction between experts and non-experts. But I have a hunch that we sometimes use this as an excuse for not looking into certain issues. For example, as highlighted in Superforecasting, normal people can form well-informed predictions on certain issues with a bit of practise. Anyway, I think there’s a middle ground here: if you don’t know all the technical details, just adjust your confidence levels.

It’s worth emphasising that forming opinions can be very difficult and time-consuming. For example, predicting technological progress is notoriously hard. It requires you to do your homework, researching which factors influence scientific advancements and so on. This goes against the idea of opinions coming to us “naturally”, as if by chance. Yet, this isn’t a good reason not to actively seek out opinions. Sometimes, we have to make our minds up in order to take action.

As long as one has some “epistemic etiquette” - being prepared to change your beliefs in the light of new evidence, and not taking everything so personally - having more opinions seems like a good thing. I’m currently trying to build the habit of always having a working hypothesis whenever I learn something new. Having more opinions makes you feel more like part of the world, rather than as a bystander.

Useful thought experiments

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 02 Mar 2025 00:00:00 +0000

Philosophers love thought experiments. Thought experiments are hypothetical scenarios meant to tease out our intuitions about an argument or theory. For example, here’s a classic thought experiment, due to Robert Nozick:

Suppose there were an experience machine that would give you any experience you desired. Superduper neuropsychologists could stimulate your brain so that you would think and feel you were writing a great novel, or making a friend, or reading an interesting book. All the time you would be floating in a tank, with electrodes attached to your brain. Should you plug into this machine for life, preprogramming your life’s experiences?¹

Many thought experiments seem kind of cooked up, so it’s easy to believe thought experiments have no practical use. However, one of my key takeaways from The Scout Mindset by Julia Galef was that thought experiments aren’t just diversion for people with too much spare time. In fact, she argues, certain thought experiments can help us think more clearly about decisions we face in everyday life². In this essay, I’ll go over three interesting thought experiments from her book, and then describe two personal faves.

The Outsider Test: This all comes down to having an outsider’s perspective, as if one were calling a friend. How would an outsider evaluate the situation? For example, say John Doe has been given two job offers: one at company X, the other at company Y. Company X will look better on his CV, but he’s unlikely to enjoy the day-to-day tasks. In contrast, the job at company Y, although not quite as prestigious, seems more fun. Here, an outsider may say something like “If prestige weren’t a consideration, which option would you pick?”
The Conformity Test: This is a good one. We’re often quick in adopting the beliefs of people we respect. And this is usually a good thing: life is to short to overthink everything, and we’ve got to form opinions somehow. (For fans of Thinking, Fast and Slow, this is System 1 in action.) However, when it comes to more delicate subjects, this mental short-cut might fail. In the Conformity Test, Julia Galef asks you to imagine that people no longer hold your view. (To all contrarians out there, just imagine that the people in your community suddenly become just as everyone else.) There’s a particularly interesting spin-off here: what if one of the main proponents of your view, perhaps the person who helped shape your beliefs about the subject, would change their mind? I think the EA/rationalist community provides a good use case. What if Will MacAskill would say he was completely mistaken about longtermism, rejecting the idea altogether? Or if Eliezer Yudowsky would declare that AI after all isn’t that big of a threat?
The Status Quo Bias Test: The underlying idea here is that humans have a bias towards the status quo. If you were to start from scratch, would you actively choose your current situation? For example, imagine a medical student who realises halfway through second year of med school that medicine isn’t for her. Although she cannot imagine herself as a doctor, she’s still hesitant to switch subjects. Here the Status Quo Bias Test might come in handy.

After reading The Scout Mindset, I soon realised that some of the advice I’ve received over the years can be rephrased as thought experiments. Here are two such thought experiments which I’ve found particularly useful:

Worst Case Scenario: This just involves asking oneself about the worst-case scenario. Is it really that bad? If yes, well, then you know. At least your fear isn’t some sort of vague, scary illusion. And if not, good!
Fast forward a decade: In ten years from now, which decision is one more likely to regret? Humans are famously bad at long-term planning. We’ll often fail to take the route of action with benefits in a distant future. (Hence climate change and the fact that most adults don’t get enough sleep.) So it might be a good idea doing some kind of Outsider test, where the outsider is one’s future self.

Some of the above thought experiments might sound familiar. Perhaps you’ve already used some of them yourself. After all, these seem like obvious tricks for seeing this for what they are. But perhaps it’s helpful having names for these tricks. It’s a bit like building a toolkit for better decision-making. I’ve applied something like Worst Case Scenario a bunch of times, but only after spending a couple of days of dwelling on the issue in a very unproductive way. Thinking in terms of thought experiments would have spared me a lot of headache.

Nozick, Robert, and Thomas Nagel. Anarchy, state, and utopia. Vol. 5038. New York: Basic books, 1974. ↩︎
Galef, Julia. The Scout Mindset: Why some people see things clearly and others don’t. Penguin, 2021. ↩︎

Hunting dependencies

ronalds.vilcins@gmail.com (Ronalds Vilcins) — Sun, 23 Feb 2025 00:00:00 +0000

One of the benefits of attending lectures is that lecturers tend to give some really good unsolicited advice every now and then. Last semester, a professor of ours digressed to talk about the importance of identifying the key ingredients of a result. He’d just concluded a rather long proof, and was about to clean the blackboard as he said that the result was elementary. See, you only need this one lemma from point-set topology (admittedly a bit niche, but easy to prove), and then the definition of the Fourier transform. Put that way, sure, maybe it’s elementary. Explicitly writing down the dependencies of an idea, he said, was a good exercise.

An example #

I tried doing this a couple of times for definitions, theorem statements and proofs. At first, it felt a bit silly. Once I finished writing down the main components of a result, it seemed trivial. I overdid it at first, writing down dependencies even for minor lemmas. But for more complicated theorems or involved definitions, it proved quite useful.

Now for a concrete example. Suppose we’re trying to understand martingales. Martingales can be thought of as sequences of random variables representing fair games: if we’re given the value of the $n$:th random variable, we expect the value of the $(n+1)$:th random variable to stay the same. There’s no predictable up- or downward trend. Here’s the definition from Le Gall:

Let $(X\_n)\_{n \in \mathbb{Z}\_+}$ be an adapted, real-valued random process, such that $E(|X\_n|) < \infty$ for every $n \in \mathbb{Z}\_+$. We say that the process $(X\_n)\_{n \in \mathbb{Z}\_+}$ is a martingale if, for every $n \in \mathbb{Z}\_+$, $E(X_{n+1}|\mathcal{F}_n) = X_n$.

First, we need some terminology related to random processes, understanding what’s meant by an “adapted process” and a “filtration”. Apart from that, we need a solid grasp of conditional expectations. The definition of a conditional expectation with respect to a $\sigma$-algebra, as well as the underlying intuition. And that’s about it.

Should I bother? #

If you want to learn something thoroughly, then yes. Tracking down dependencies makes you engage more with the material. Learning a concept isn’t just a matter of being able to regurgitate the contents of the lecture notes - it requires you to build your own mental model of what’s going on. As I see it, the purpose of exercises, quizzes or review questions is to make us think more carefully about a given topic. Otherwise, because humans (or at least maths students) are lazy, chances are we’ll go through the material too quickly. Tracking down dependencies is a bit like doing more problems, in that it prompts us to revisit the material.

Another benefit of nailing down dependencies is that it makes the concept seem relatively simple. Some results can seem quite daunting at first. But working out the dependencies can make the concept seem deceptively simple - almost to the point where you’re struck with the curse of knowledge. Moreover, if I struggle to understand an idea but am clear about the dependencies, I know what to do: I’ll just read up on each of the topics involved. In this way, the dependencies translate into an a checklist for my learning process.

Lastly, I found it satisfying seeing how different notions tied into one another. I also put it all into one mindmap which, apart from having a high aesthetic value, gave me the big picture of the subject.