I am curious what happens when the accumulated network or body of knowledge that is grown in this manner can be corrupted by what is left out... I have experiences in research using AI concerning fringe concepts, that may be supported as anecdote, but ultimately these aspects are glossed over or dismissed for the more repeated or common thread. So biases are reinforced. New understandings are pushed to the periphery and largely dismissed. This matters when much of what we understand about out world has been built on lies and manipulated towards consumerism and extractive technologies. Curious your thoughts or experiences of this, ie... AI hallucinating etc.
Brilliant piece! Haven't read a perspective that breaks down LLM knowledge bases in a humanistic level like this. You bring the relevancy of this technical topic to the non-technical readers in a wonderfully articulate manner.
I've been building and using the hybrid system for the past two years.
We augment ourselves by processing information faster (Karpathy style of dump raw data and synthesize) but compound our personal abilities by doing the writing and thinings (Luhmann style).
It's a nuanced balance I've not seen many other folks talk about.
I'd like to see a world where we use AI to amplify our abilities not just to produce more output.
Next thing Karpathy will find out important will be intentionality. Then, that intentional outputs can shape the pipeline instead of inputs. Then, that pressure (external or internal) to get these outputs out of the system can be deliberate and managed.
The core reason Luhmann was productive was not Zettelkasten. It was the pressure. The external – publishing obligations, institutional accountability – or an internal one, but something that made the system necessary; and constantly reinforced it.
Some people copied the box without the gun to their head.
Amazing breakdown on the similarities and differences between Andrej Karpathy's LLM PKS and Niklas Luhmann's Zettelkasten. I've been thinking about how best to incorporate AI with a second brain and the last section about them making different bets on what a PKS solves was really useful. Explains why I feel like most AI workflows goes against the purpose of a Zettelkasten.
I tentatively think I'm in very substantial agreement with this. I'd offer a synthesis (which might map closely to the one given).
Automated mapping *of discourse* absolutely has its place, and in fact could enormously accelerate individual learning as well as the ability for interlocutors to get on the same page, at least about *what has been said, what arguments and positions there are, and how they relate*.
Notably, exemplary encyclopedia articles, literature reviews, and surveys do not present single perspectives, even on 'settled' topics. On settled topics, you are presented with a chronology: how did we get here? What *were* the arguments? How were they interrogated, litigated, approximately settled? Meanwhile, on 'live' topics, you see the array of positions, arguments, and their (ideally best) cases and rebuttals. Really, Wikipedia articles at their best often have this structure to them!
That's where automated knowledge-base creation can really shine. It oughtn't prematurely condense, filter, or synthesise. Hold space for the living and historical discourse.
Then the individuals (and sometimes AI!) who want to become deeply acquainted with a topic can progressively peruse such maps, interrogate them, digest them, add to them. That's part of the hormetic process of becoming equipped with the context to add more. Automated, as well as human (and mixed-mode) *surfacing of candidate gaps* or *candidate underexplored connections* is certainly as fruitful and of shared benefit as it's ever been.
The process of generating *new contributions*, either new positions, counterarguments, new syntheses - that comes from a place of familiarity with (at least some of) what exists, together with familiarity and fluency with other concepts and experience - it can only be *better* enabled by having more access to good mapping of topics and discourse. (This too could be done in part by AI some day.)
I liked this article a lot, but the unexpected pleasure was in nodding my head along with the specific examples of biosemiotics/unicellular cognition you cite, since that’s a topic I am actively curating and researching….
The hormesis framing is the clearest I have seen for why the friction matters. I have been writing about the same architecture from the practitioner side, what it actually feels like when you are in the loop versus when you have quietly handed off the thinking without noticing. The moment you realize the debt has accumulated is always later than you think.
I am curious what happens when the accumulated network or body of knowledge that is grown in this manner can be corrupted by what is left out... I have experiences in research using AI concerning fringe concepts, that may be supported as anecdote, but ultimately these aspects are glossed over or dismissed for the more repeated or common thread. So biases are reinforced. New understandings are pushed to the periphery and largely dismissed. This matters when much of what we understand about out world has been built on lies and manipulated towards consumerism and extractive technologies. Curious your thoughts or experiences of this, ie... AI hallucinating etc.
This is awesome and so closely aligned with what I’ve been thinking about and thinking about what to build.
Brilliant piece! Haven't read a perspective that breaks down LLM knowledge bases in a humanistic level like this. You bring the relevancy of this technical topic to the non-technical readers in a wonderfully articulate manner.
I've been building and using the hybrid system for the past two years.
We augment ourselves by processing information faster (Karpathy style of dump raw data and synthesize) but compound our personal abilities by doing the writing and thinings (Luhmann style).
It's a nuanced balance I've not seen many other folks talk about.
I'd like to see a world where we use AI to amplify our abilities not just to produce more output.
Great post.
Good to hear. Did you use Obsidian for the personal writing or some other software?
I’ve been using Obsidian for the past 3 years for my personal wiki. Before that was a decade+ in Google docs.
However, when I actually am writing a draft/ editing I’ll move the post to a Google doc- the comments make it easy to review.
Next thing Karpathy will find out important will be intentionality. Then, that intentional outputs can shape the pipeline instead of inputs. Then, that pressure (external or internal) to get these outputs out of the system can be deliberate and managed.
The core reason Luhmann was productive was not Zettelkasten. It was the pressure. The external – publishing obligations, institutional accountability – or an internal one, but something that made the system necessary; and constantly reinforced it.
Some people copied the box without the gun to their head.
great article! i believe that AI productivity comes with hidden cost of intelecutal decline that will eventually pop up in a longer run
This might be the next step in human literacy - knowing how to utilise the LLMs in a sustainable way.
Amazing breakdown on the similarities and differences between Andrej Karpathy's LLM PKS and Niklas Luhmann's Zettelkasten. I've been thinking about how best to incorporate AI with a second brain and the last section about them making different bets on what a PKS solves was really useful. Explains why I feel like most AI workflows goes against the purpose of a Zettelkasten.
I tentatively think I'm in very substantial agreement with this. I'd offer a synthesis (which might map closely to the one given).
Automated mapping *of discourse* absolutely has its place, and in fact could enormously accelerate individual learning as well as the ability for interlocutors to get on the same page, at least about *what has been said, what arguments and positions there are, and how they relate*.
Notably, exemplary encyclopedia articles, literature reviews, and surveys do not present single perspectives, even on 'settled' topics. On settled topics, you are presented with a chronology: how did we get here? What *were* the arguments? How were they interrogated, litigated, approximately settled? Meanwhile, on 'live' topics, you see the array of positions, arguments, and their (ideally best) cases and rebuttals. Really, Wikipedia articles at their best often have this structure to them!
That's where automated knowledge-base creation can really shine. It oughtn't prematurely condense, filter, or synthesise. Hold space for the living and historical discourse.
Then the individuals (and sometimes AI!) who want to become deeply acquainted with a topic can progressively peruse such maps, interrogate them, digest them, add to them. That's part of the hormetic process of becoming equipped with the context to add more. Automated, as well as human (and mixed-mode) *surfacing of candidate gaps* or *candidate underexplored connections* is certainly as fruitful and of shared benefit as it's ever been.
The process of generating *new contributions*, either new positions, counterarguments, new syntheses - that comes from a place of familiarity with (at least some of) what exists, together with familiarity and fluency with other concepts and experience - it can only be *better* enabled by having more access to good mapping of topics and discourse. (This too could be done in part by AI some day.)
https://www.oliversourbut.net/p/a-full-epistemic-stack
I liked this article a lot, but the unexpected pleasure was in nodding my head along with the specific examples of biosemiotics/unicellular cognition you cite, since that’s a topic I am actively curating and researching….
The hormesis framing is the clearest I have seen for why the friction matters. I have been writing about the same architecture from the practitioner side, what it actually feels like when you are in the loop versus when you have quietly handed off the thinking without noticing. The moment you realize the debt has accumulated is always later than you think.
https://cdaemon.substack.com/p/the-mirror-that-talks-back?utm_source=share&utm_medium=android&r=7w111a
https://nandigamharikrishna.substack.com/p/andrej-karpathys-llm-wiki-full-breakdown?utm_campaign=post-expanded-share&utm_medium=web
Thank you for this article, I just starting my digital garden.
Very helpful comparison to consider
Great insight no LLM could synthesize if we give them these three tools and approaches.
PhD student here, I’m wondering if I can integrate this with a reference manager such as Zotero