Marginalia
Viktor Lofgren
My name is Viktor. I’m a Swedish software engineer and hypertext enjoyer. Marginalia is a website I’ve built. It’s really almost a bunch of websites on a common theme. If you find yourself clicking a link and ending up on a page that looks completely different, that’s just how things are.
July 2026
Top Words
- marginalia
- search
- index
- software
- years
- grant
- crawler
- astrolabe
- internet
- query
All posts (137)
One of the more common feature requests I’ve gotten for Marginalia Search is the ability to search by date. I’ve been a bit reluctant because this has the smell of a a surprisingly hard problem. Or rather, a surprisingly large number of easy problems. The initial hurdle we’ll encounter is that among structured data,...
By which I mean there are deeply problematic assumptions in the very notion of scaling: Scaling changes the rules, and scaling problems exist in both directions. If what you are doing effortlessly scales up, it almost always means it’s egregiously sub-optimal given your present needs. These assertions are all very...
A very brief note to announce reaching a long term goal and major milestone for marginalia search. The search engine now indexes 106,857,244 documents! The previous record was a bit south of seventy million. A hundred million has been a pie-in-the-sky goal for a very long time. It’s seemed borderline impossible to...
In the primordial days of Marginalia Search, it used a dynamic approach to crawling the Internet. It ran a number of crawler threads, 32 or 64 or some such, that fetched jobs from a director service, that grabbed them straight out of the URL database, these jobs were batches of 100 or so documents that needed to be...
I discovered someone has made a cryptocurrency called “Memex Marginalia Inu”. It appears to have been created February 23, which is around when the entry “I Have No Capslock And I Must Scream” went absurdly viral to the point where Elon Musk tweeted a link to it. I Have No Capslock… Mr Musk’s twitter orbit is...
Bots are absolutely crippling the Internet ecosystem. The “future” in the film Terminator 2 is set in the 2020s. If you apply its predictions to the running of a website, it’s honestly very accurate. Modern bot traffic is virtually indistinguishable from human traffic, and can pummel any self-hosted service into the...
I’d like to discuss a mental somersault that I’ve found has caused me a lot of grief in the past, which is prescriptive descriptions. Let’s break this down a bit: A descriptive statement is a statement about how something is. A prescriptive statement is a statement about how something must be, a rule or a law. If I...
Anchor texts are a very useful source of keywords for a search engine, and in an older version of the search engine, it used the text of such hyperlinks as a supplemental source for keywords, but due to a few redesigns, this feature has fallen off. Last few days has been spent working on trying to re-implement it in...
After a bit of soul searching with regards to the future of the website, I’ve decided to open source the code for marginalia.nu, all of its services, including the search engine, encyclopedia, memex, etc. A motivating factor is the search engine has sort of grown to a scale where it’s becoming increasingly difficult...
There are a lot of ways of building software, there are many languages you could choose to build it with, many libraries to rely on, many frameworks to leverage, many architectural approaches, many platforms to choose, many paradigms of daily operations to follow. It takes years to get in-depth experience with just...
Page 8 of 14