AI operational knobs and strategic levers

Today's core technical reasons why changes will not slowdown

While looking at the news reaching the masses worldwide and hearing people talk about AI (including tech people), it is hard not to notice how unaware most people are of what’s going on; at the same time, I understand how challenging it is for most to grasp. Plus, the current communication stream noise is absurd.

In a nutshell, what most people don’t see, understand, or realize about where we are in the development curve of AI is the fact that the changes we are seeing (and will continue to) are associated with the inherent sheer number of operational knobs (technical and hyperparameters of deep learning architectures) and the strategic levers available to play, experiment, expand, and advance, leading to more advanced features and capabilities. So the fact that things are changing this fast isn't a big surprise if you understand how many factors are at play at a time.

Figure 1: illustration of the many strategic levers inside frontier AI labs. The dot over the line is meant to give an idea of where we are on the curve today (at the time of writing this). In the image, the strategic levers are split into blue and red. The blue ones are more or less under engineers’ control, and they “understand” them, while the red ones show side effects that research engineers still don’t have a clear strategy to handle as the blue ones keep evolving.

At the time of writing this piece, if we pick the “recent” HuggingFace (HF) attack event that was widely publicized everywhere (news media, influencers, all social networks, etc.) and put it into perspective with where things are on the curve of AI development, we will have something like Figure 1. In it, you can see an approximation of where we are and the direction things are heading today¹. It includes a marker identifying the HF attack on a few selected key strategic levers being experimented with and advanced. Keep in mind that this is just a sliver of what’s being actively experimented with; there are many more active research areas, architectures, and techniques. And the challenges are as many as the opportunities to further boost what we already have today, e.g., interpretability, alignment, abliteration, safety and regulation, security, memory, etc.

Buckle up; a lot will keep changing.

Footnotes

1. By the way, at the pace the field is moving at right now, these approximate representations could even differ by orders of magnitude in 1 day, a week, or a month.