Came across this blog today and this paragraph amused me a little.

A potentially reasonable reaction to these limitations would be to say “LLMs have improved a huge amount over the past year and thus future improvements over the next year will likely make them much better at writing good documentation and maintainable code.” But if you accept my staircase hypothesis, that reaction is a lot less certain. Being able to climb up a tall staircase doesn’t mean you can swim.

This is true! I see AI pilled engineers using LLMs to generate arbitrary documents and share with others without verifying the veracity or worse yet, whether the information is for the intended audience.

A little judgement goes a long way.

I’ve always been fascinated by the requirement for “unification” and “a single thing” to answer all questions in the western world. The western world has embraced monotheistic regions in the last 3.5 millenia, if you trace back to the origins of Judaism. While Europe and likely the Americas were rich with pagan gods, we all know how fast monotheistic regions have taken over.

I’ve likened this too physicists search for a single Grand Unified Theory and I’ve thought westerners have the need to explain everything with a single thing. This is repeating in the world of AI too. There’s a search for the largest SoTA model that can do everything.

Coming from a different world view, I can’t understand why the world has to be so simple! There are ~8 billion people in the world and everyone has a different world view. And the more diverse the world, the more rich and amazing it is.

The more models we have, in varying sizes, the better it is for the world!

At times when I feel incredibly frustrated or fatigued that I feel I should just become a recluse, I think of someone like David Attenborough. He turned 100 a couple months ago and still carries on with the same enthusiasm that he had when he was 20!

Sure, he’s physically much weaker but mentally astute and curious. I was reading about him on Wikipedia and came across this quote

If I was earning my money by hewing coal I would be very glad indeed to stop. But I’m not. I’m swanning round the world looking at the most fabulously interesting things. Such good fortune.

The world is incredibly beautiful and amazing and one just has to find beauty. This is easier said than done though.

One of Descartes’ foundational contributions to Philosophy is cogito, ergo sum which translates to “I think, therefore I am”.

LLMs are fascinating and helps you learn and understand things a little more by sort of pointing you at the right direction if you know what you’re looking for. In a world where we’re optimizing for speed and max throughput where Software Engineers are purely acting as a conduit between an LLM and their systems and stop thinking, are we losing a part of our humanity in the process?

Life is incredibly rich! And we owe this richness to the differences that exist between us as a society! We don’t expect us to wear the same clothes around the world, eat the same food, like the same movies and speak the same language.

One of my biggest fears of an LLM driven society is homogenization. LLMs are an incredible capsule of our society. We have a lot of information distilled in these models that help us already day-to-day. But systems being developed look incredibly alike - we’ve seen this in the UI mockups that LLMs generate!

How would it look if every building in every city looked the same - this is already happening in modern architecture. The richness of gothic art and the awe-inspiring gopurams of India are largely from a bygone era.

How do we make sure that we don’t homogenize society as a whole as we embark on more and more use of LLMs?

Cory Doctorow termed this word in 2022. This is a word that represents the state of much of software today. Not just the end state, but often intermediate states and workflows/processes that lead us there. And I fear LLMs without the right mindset might get us there sooner.

While an abundance mindset is good, it’s often the decadence mindset that leads us there.

On a daily basis, I come across folks who just prompt the LLM to run simple bash commands that’d at the most take you another couple mins etc. but in the process, you learn something. And there are tools like butterfish, while being incredibly cute, can easily push one to a mindset where you don’t learn what happens underneath!

And then there’re folks who essentially use the LLM to churn 10x output. But output not necessarily translates to outcomes long-term. But in an environment where your worth is determined by the output in the near-term, one has to adapt to using these tools else the 10x output engineer is going to get the early worm.

Only time will tell if this will last.

I read this fascinating post by Armin Ronacher on the perils of using agents. This is something that hardly anyone bats an eyelid to - the rationale is if something is messy, I’ll throw it out and write it again.

We are going to enter weird times where we’ll have an army of new engineers who are likely to know how to prompt but not really how to think. Some senior engineers might end up here too.

The best thing one can do is to be active and vigilant and actually bring some thought into building things. Define design choices and ask the LLM to adhere to it. It’ll likely not do it from time to time and one should steer the model to do the right thing.

Else, we’ll all be drowning in slop soon.

TIL about Hinton’s Paradox where Geoffrey Hinton’s prediction in 2016 that we won’t have any more radiologists in 5 years isn’t just true but also that we have now more radiologists than ever. And of course, AI has integrated into every workflow of radiologists.

And this reminds me of another fantastic article that I read last year about how Cancer has a surprising amount of detail by @owl_posting. The world has fractals all around us. As we zoom in further and further in whatever we do, we get specialized and we collectively improve what we as a species can do and our expectations of ourselves get better. AI helps address the grunt work and a smart human uses AI to automate the grunt so that they can focus on the more important stuff to advance ahead.

Those are the people that survive.

Adam Wiggins, the founder of Heroku announced today that he’ll be researching on how to combat the information overload that we all have to deal with on a day to day basis as part of his time at Ink & Switch. I’m super stoked about this tbh. Adam is an amazing researcher with a keen eye for solving problems elegantly and I’m looking forward ot seeing how this evolves.

Admittedly, this is controversial but I’ll die on this hill, nevertheless.

People have long careers and work for a bunch of companies over time. A lot of people put the ex-$prev_company tag on their profiles. I like to think that they do it for visibility and to boost the possibility of their profile showing up to recruiters (if they’re searching for new roles). I don’t really blame them - it’s the nature of the game and you have to play the game that the algorithm mandates.

However, it’s not a surprise that some of us do it for virtue-signaling.

My opinion is you’re doing more long-term damage to yourself and your career by having an ex- in your tagline. Sure, you should include that in your CV or experience but you don’t have to define yourself by it.

Companies are stints, your individualism matters and who you are and what you bring to the table matters more than the fact that you were an employee of Google, 5 years ago!