Non-determinism in LLMs
Everyone knows that LLMs are inherently non-deterministic and harnesses, MCPs, RAG, skills etc. all try to paper over this.
But this non-determinism is a feature. And the developer community is slowly beginning to embrace it. This was well elucidated by Martin Fowler on his recent podcast on The Pragrammatic Engineer. He akins this to the field of mechanical engineering with engineers building tolerances into what a structure can withstand.
Computer Science Engineers actually have an example closer to home. Not too long ago, Google ushered us all in with the MapReduce paper and the core of this architecture was GFS which inherently assumes that all underlying hardware components are unreliable. And this has prompted the industry to build strategies like replication, fault-tolerance, SRE practices around the fact that systems are fallible.
Perhaps, our industry can consider this as a watershed moment before the mother of all uncertainties becomes more mainstream - Quantum Computing.