Ask HN: How Do You Utilize Your Personal Knowledge Base?

I used to be deep into the personal knowledge management tar pit. Most of it was wasted time. Here are the few things in it that I actually regularly use and get something out of. These are just simple zim-wiki text files that I update and skim through a couple times a month:

- Idea list: Ideas for art or other projects that I'd like to try some day. I'm occasionally adding new ones and I've started work on old ones I've added that finally piqued my interest.

- Gift idea list: Things friends and family may hint at wanting or outright say they want in casual conversation. I add stuff to this list then buy it for birthdays, Christmas, etc. Can get some real nice surprises for people when they only mention things in passing and you happen to catch it.

- Book list: Just a wish list of books. Occasionally I'll buy a few to add to my personal library and cross them off the list.

Just understand that most of the personal knowledge management stuff is a waste of time. It's a bunch of feel-good busywork that's typically sold to you first as a cheap idea, then sold to you as products like Roam Research, Obsidian, etc. Can't really beat simple lists in text files or on paper. I even like physical notebooks that you go through and keep a hand-written index in the back of. Forces you to think and revisit stuff.

I use chatblade, Obsidian, and git. I generate a diff, send it to GPT-4 which gives me a short description of what I changed and then I push to github. This makes maintenance easier.

You can do it automatigally, but that can get expensive rather quickly.

I use it as the means to understand things. I write as if I am explaining things to somebody else, that else is the future me, but in the process you can quickly understand what is not clear in your head, break it down to first principles, and build it back up again.

N=1, but writing has had an enormous impact on my growth.

I keep everything on a wiki I run on a home server.

Over the decades, I think I've tried just about every method of maintaining my knowledge base, project notes, to-do lists, etc. And the only one that works even halfway decently for me is a wiki.

It's searchable, the simplicity of markup means I won't get side-tracked into wasting time on layout and eye candy, it's easy to search, if I have internet access then I can access it from anywhere, and the wiki I use allows simple embedded spreadsheets and other such functionality for more complex data.

Plus, it's very easy to allow others secure access to it for when I want to share notes, or keep notes that are updated by multiple people (such as when collaborating on a project), or to distribute very large files.

I started using Foam[0] a few years ago, but the more I used it, the more I dropped all the tedious bits, and it became nothing more than a big, evolving markdown repo.

I switched from vscode (back) to vim, and it has worked as well or better than it did before. I follow my own rules. I like the Zettelkasten idea of one idea per card, but if I put more related things in the same .md file, that's OK. I didn't like the flat directory structure, and so I have dirs organized by category. My /bar directory is inside my /cooking directory, and for whatever reason, that makes sense to me. Ripgrep doesn't care, and I always find what I'm looking for.

This markdown hierarchy, that still lives in a repo called "foam", has become indispensable to me.

[0] https://github.com/foambubble/foam

The secret sauce is orgmode in emacs.

My personal notes are a .org file that is backed up on a private GitHub repo.

I have sprawled about 5k lines at this point since around 2020.

Some things get a org-bullet (*) and a TODO.

Then I just do org-agenda and it pulls up a nice view of all my TODO items for pruning and prioritising.

Having an automatic agenda and the fact the bullets are tab-collapsible and can be used as an outliner is the only way my notes are not out of control.

My other writing is in apple notes (short lived) or exists in tickets. I’ve tried various things like zettlekasten and obsidian but nothing sticks for me.

My other daily task setting and personal planning is a bullet journal with a future log and monthly logs. Bullet journaling is how to get it all out and process and plan personal things. Actual Knowledge is in the org-mode.

For email I follow getting things done and have an inbox an @action and a @waiting folder. I keep inbox empty and the others get pruned every day.

Funny how often the word recollect has come up here, without mentioning the fantastic tool recoll, updated frequently and available for many operating systems. I use it on macOS for an immense library of scientific and other documents. I think there's a standalone installer, and it's also available via macports. It may be found at

https://www.lesbonscomptes.com/recoll/pages/index-recoll.htm...

It allows me to index and find words and phrases in files of many types AND view their contents independently of external readers, or using system readers, including text, PDF, or word processing files of common sorts.

Thanks for sharing --

> More importantly, the process of adding to and maintaining the knowledge base felt more burdensome than beneficial

I wonder if being able to recollect things later in an organized way may not be worth the effort for you.

Something I've found is that I really don't need a robust personal knowledge base. My goal is to do my work well (and taking notes to support this is definitely valuable). Building a personal knowledge base doesn't support this goal and has a more unclear value-prop, so it's a drag to keep up with.

Wrote an opinion piece on this https://www.stashpad.com/blog/you-probably-dont-need-a-knowl...

My motivation was multi-sided and I went with a completely ad-hoc structure. I store different kinds of information in very different ways. I sometimes reorganize a section. I don't use any monolithic or gate-keeping tool - the information and data is far more important than the tools. The oldest sections go back about 30 years. I have a few conclusions:

(1) It takes time to reach critical mass. Until then, it's overhead. But even then some content was useful from the very start.(2) After that it became very reliable - it's there and I rely on it. (3) For me, it's clearly not just knowledge base, it's PIM - personal information management - KB is just mixed with the rest. (4) After decades, retrieval is still a mix of ideas and tools - not great. (5) I still wish for more efficient data entry / collection.

In balance, for sure a majority of the content is never accessed.That's fine. The system allows un-foreseen uses to happen commmonly and I like that. Do the positives outweigh the costs over the years? I have no idea. I add stuff constantly, some things I rely on again and again, some query ideas are one offs... and usually they work which is great. I know I have a very solid base for recollection. It's not clear how I would do a cost/benefit analysis.

I feel perfectly fine with the ad-hoc collection and storage strategy. That has worked well enough.

I use Obsidian to manage personal knowledge about apps, OS features, and workflows. Some of it has to do with personal configurations; some of it has to do with undocumented behaviour I found out myself, issues, or coalescing fragmented information from around the web. I'm pretty careful to not simply rewrite what the web already documents well or is easily found in a single search. So it is a mix of official documentation links, written knowledge, and personal experiences like the search query paths I used.

The approach is basically "I write what I've used if I still have around what caused me to need it. I associate what I use with where I access the context of the cause". Because issues are context dependant you organize by the context that it came from. If you name a file as the app name you can have the document come up when you search for the app name. Because the content is based on situations and issues, returning to it is predictably probable. I find this is a pretty low mental burden to writing, maintaining, discovering, and navigating. You don't really have to make decisions about where to place something, how to name it, or how to find it because it follows from naturally arising circumstances and contexts.

I don't think I technically need to use a PKMS as I use the special features lightly but it is nice to have them there when you really want them and to have an interface dedicated and designed to deal with servicing the whole workflow. I also like to write down my stream of thoughts too but I keep them separate and use a separate system for interfacing and managing them.

Use of this type of thing has been absolutely essential to my work as a research scientist, as the modern equivalent of a 'commonplace book' (https://en.wikipedia.org/wiki/Commonplace_book). Search is the primary advantage over paper notebooks and so search is arguably the most important feature of any of these 'knowledge bases'. Tagging or other forms of organizing being subsets of search, and in my opinion essential features.

So, how do I employ it? One is to search on a subject I'm interested in or working on, ie to find notes, previous readings and writings, then add to these or go from there. Another is to keep various kinds of lists, (eg lists of ideas, things to read, etc). Otherwise the way your able to link things together with tags and other forms of linking is also pretty great - these are all features I will insist on when I move to something new, since I use tiddlywiki, which is a mostly dead project.

Anything with these features works like an external brain, allowing for recall, with just enough inefficient that you have to engage with the material, and possible mix some notes and facts together to hopefully inspire some new ideas.

I run a mediawiki instance at home (a few containers managed with docker compose) and it's been great for the last 3+ years.

There are a number of feature for organizing content (the main ones being Categories and Namespaces) and it has APIs (so i can write simple python scripts to perform actions -- like updating some index pages with a custom format).

The VisualEditor is very nice and it's fairly easy to embed images, videos or even pdf documents.

It's not exposed publicly but I can access it (both for viewing and editing) from anywhere using my own vpn connection.

I update the main software once in like 6-8 months so it's fairly low maintenance (this boils down to essentially changing the version tag in docker compose file, restarting and checking everything's working alright -- with the occasional fixes).

Since it's running at home (on a small computer that i would keep on anyway and that it's also doing other stuff) it's essentially free and i don't have to worry about space or any other metric.

I have a smart RSS reader I am working on called YOShInOn. The main UI looks like TikTok or Stumbleupon, I can thumbs up, thumbs down or favorite articles, out of maybe 2000 articles a day it picks maybe 300 to show me, maybe I really read 200 on the average day.

Once the articles get favorited though there is the problem of navigating them, I just added some simple text search capability but there is a huge amount of room for improvement there. If it gets the ability to ingest articles via bookmarklet than it becomes a bookmark manager, add some ability to enter notes locally and it becomes a more general knowledge base manager.

It could really use "tags" but when it does these will be tags for the ML age, particularly it will be possible to have positive, negative and indeterminate tags. If the ML system thinks a tag applies it goes into indeterminate status and could be set to negative or positive to make it a training example.

My unit of information is "the idea" and they all go into a single markdown file with a Markdown heading. The ideas are numbered and I use internal page GitHub anchor links to link between headings.

I had a WikidPad Wiki (python desktop based wiki software for Windows and Linux) that has 1000 entries. There was a number of pages that were long. But wikilinks in that wiki were often definitional (rather than explain inline, they had their own page) and multiple pages linked to them. The homepage of the wiki is a launching pad for links to other pages.

That wiki is a design of my ideal computer system, it's on GitHub as idea-wiki.

I think I prefer the all in one file thing because I don't need lots of very small files with very little information in. I also read over my notes regularly.

If you want value from your notes I feel you can read over them regularly.

I had a similar issue. I had read How to Take Smart Notes. This discussed the slip box method. I was recording ideas in Obsidian. But I was not reviewing them.

Fast forward another year, I saw a mention of a book called Building a Second Brain by Tiago Forte. I bought it and read it in 3 sittings.

The concept was that you are only capturing what resonates with you. Stuff that is actionable for your projects.

The structure of your folders helps with this. So current projects are easy to locate. You can keep working on them.

He suggests in a bonus chapter on tagging to just use the search mechanism of your app. He gives some other useful ways tags can be used.

Fast forward to today, I am actively reviewing my notes and improving them each time I touch them to make them more useful.

I keep all my zettles at https://wiki.malloc.dog/.

When I was doing my masters in anthropology at the same time as my SWE job, I ran into this issue of having too many notes and not enough organization. Random other tools never quite did it, generic search didn't work, notion et all kind of sucked, so I ended up rolling my own thing on top of Org roam. It's all plaintext org files, easily accessible.

When I do scholarly work, it's great because all of stuff is in one place. It's also useful for tracking the breadcrumbs I do for my real tech job.

I also collect these constructs. It's an obscure science; one with a founder; C. Wright Mills. I'll honor his work by using his term: "Intellectual Craftsmanship". This is covered in his book, "The Sociological Imagination." It appears as the appendix of the book, and a website can be found at the site below. You will also find in Google results that there are multiple courses of this name in sociological studies. http://www.sociological-imagination.org/

The only practical use I've found for knowledge base / bookmarks is to have a searchable list of references/docs/cheatsheets/examples that are hard to find via google (or are not available on google,i.e. internal company docs).

With the release of ChatGPT my need for a knowledgebase has decreased by 10x. And I assume that the need for knowledge base/bookmarking solutitions will disaapear completely in the next few years as LLMs improve.

I use Joplin and I am playing with emacs orgmode. So far Joplin has been a lot better at finding stuff I forgot about but knew a few letters that would be there (like ssh). I have many 1000s of notes; from emacs to notepad to ms docs to onenote to I forgot to Evernote to vim to notion to Joplin. And maybe back to the emacs. I use and have used it daily ; it makes me think clearer to write things down before implementing or discussing. I basically write down my mind voice to remember it verbatim.

I use the Standard Notes[0] free tier. I organize notes with tags. I self-actualize things with my TODO, and everyone should do that. I'm often reminded of the phrase: 'Because I said I would'.

[0] https://standardnotes.com/

I have my photos and videos in a tree structured by yyyy/yyyymmdd/ with directories created by the camera under that. This helps me remember what I was doing and when.

I keep a WikidPad wiki locally. In the past I used word2000 outline mode for logs at work.

take only text notes. folders hierarchy is categories. strictly split paragraph on meaning complete information within a subject. use string similarity vectors to assign tags, not to text lines. use tags metadata with bot.

I don't believe in a "personal knowledge base" I believe in blogging, then finding my stuff on google when I have that problem in the future :)

I use Obsidian with several thousand notes for everything "writing" (structured - studying, definitions, thinking things through, and unstructured - memorable moments, stories, people, random streams of thought). "Actionable" things (todo lists and tasks) have their own place. My motivation is quite similar - I feel like I just forget things. And it's been wonderful! It's so nice to write about a nice moment, and be reminded that it happened - small things you'd otherwise forget.

I've found ideas from Andy Matuschak extremely useful - https://notes.andymatuschak.org/About_these_notes?stackedNot...

On a practical level the crucial features of what's been working for me for a few years now (inspired by Obsidian community and Andy):

* Notes have to be very clearly "bounded" - break things into smallest logical "statement" units, and interconnect notes instead of having behemoth notes with everything in one place

* Note titles have to be an explicit "statement or sentence summary of the contents, with relevant context" - which really helps with search / discoverability. E.g. "Browsers are the most common and visually advanced applications that make use of the internet, but there are also many others" instead of "Web browsers" or "Applications that use the internet". Or "The day I embarassed myself at football". An alternative of this is to have explicitly clear headings.

* The idea of putting notes into a "reverse spaced repetition" algorithm where instead of spacing things you've learnt further and further into the future, you are spacing things that are "fruitful" nearer and nearer in the future (and when you have less to add, further into the future, when eventually a note becomes "evergreen"). You then have a "queue" (mine is always packed) of notes that are due, and you pick them up as you like. This can be achieved with https://github.com/st3v3nmw/obsidian-spaced-repetition and is described better by Andy: https://notes.andymatuschak.org/zJrfPCbY7GcpV9asEc8NTVzXTAV4...

With the above things combined, I usually sit down just to write and unload things from my mind and enter notes into the (basic) "attention algorithm". That way I never stress about polishing notes, and there's always a time in the future when I come back to a note written a long time ago, make some edits, maybe split it into a couple of notes, clarify the title, maybe merge a duplicate (if something has come to mind twice, it's clearly important to me!!). With explicit titles I almost immediately know what the note is about, trigger new connections (between the time I've written the note and now, some other relevant thing may have surfaced which changed what I think or know, or I've come up with a clearer way to describe the same thing).