NL122 - Small Models Big Folds

Hello from my home in Helsinki! This is NordLetter #122, a weekly newsletter on living and walking in Finland.

To follow the series, you can subscribe here. A new NordLetter will land in your inbox every Sunday. Not a fan of newsletters? No problem! You can read previous editions here. You can also subscribe to the RSS feed.

You can reach out to me by replying to this mail. I am also posting on Mastodon.

On Tuesday, which was earlier than I usually do this, I started thinking about what to write about in this week’s NordLetter.

Here’s how I’ve been doing this for the past few months -

1. Each day I write something in my daily-notes.
2. As I get closer to Sunday, I review some of these things
3. Around Thursday (more realistically on Saturday nights), I start a new note.
4. And then I write.

For this second step, I have used Claude (checks notes) six times in the past many years.

This was not something I found value in earlier. Of course, I could go look at what I wrote over the past week - and I do. What Claude provides me with is structure. And a single place to look at all the things that interested me over the past week.

Sometimes I know what I want to write about - maybe an event I went to, or a place or something. Those days are comparatively simpler, in a way.

Anyway.

The thing that happened on Tuesday, as I started thinking about this, was that a dread filled me completely. Perhaps it was a result of the presentation I watched last week (thanks Alex!) about AI companies using your data in whatever way they pleased. (Side note - OpenAI may have stolen a mathematician’s solution to a millennium problem.)

I have given Claude access to my personal vault. Mostly for updating metadata on notes in my vault. I wondered if a local model may be a better option for things like this. For small things I want to run in my vault.

And so, I downloaded LM Studio’s Bionic app. Then I asked Claude which models I could try out, that would run on my M1 Mac. It suggested three - Qwen3.5 9B, Gemma 4 12B and GPT OSS 20B model.

I downloaded all three. The GPT OSS model refused to load on my paltry 16GB Mac. The Gemma model got stuck at some 80% or so. I realised then that I could run only small models on my Mac. So I deleted the other two and ran the same prompt I use with Claude, to look at my notes from the last week and give a suggestion.

It took a long, long, long, long time to finish running. I shut off the monitor and went to sleep and checked the output the next day. It was nothing special.

And along with it, came a realisation that as it stands today, I would need to get a better Mac with humongous amounts of RAM (32GB minimum, preferably 64GB) for something that is productive to use.

Most of the people I know who are testing local models have received these better Macs from their companies. I haven’t. And outside of running the local models, I don’t have any need for the memory.

I am perfectly happy with my M1 Mac. And then after I was thoroughly disappointed with running the model, I wondered if Apple would make their new Siri accessible enough to run these tasks. Would it be any good?

Speaking of, Apple held its annual iPhone event on Tuesday. Among other things, they announced the iPhone Duo. And it is so damn tempting. I have found the foldable shape interesting in the past. I was briefly considering getting a Fold and leaving Apple land behind.

The Duo has all the flourishes you expect from Apple. The animation when unfolding or folding the device is just chef’s kiss.

The device I am more likely to order is the Apple Watch. They updated the internals of the things, so that the heart rate sensors poll data every five seconds. They touted a study where Apple Watch was found to be most accurate amongst its competitors. Which, of course, they would. Otherwise, the watch looks the same.

I am reading Carl Sagan’s Contact right now. It is a short book, and a book I’ve been wanting to read since I watched the 1997 movie by the same name. The book is narrated by Jodie Foster (the actress who plays the titular character in the movie).

And maybe, I have borrowed too many books from the library. Because I have some five books in the queue after this one. Wish me luck!

If you enjoyed reading this, and know someone else who might, please consider forwarding this to them. It would help this grow and make me happy. 😄

Until next week.