By the data processing inequality, we know that AI will never achieve greater information (intelligence) than what goes into it. As noted in the previous post, a local instance of AI may learn from communications with another, but the data processing inequality holds.
So, our objective is the utility of AI. In other words, to use it as a handy, accessible repository of helpful (true) information, not as a font of new knowledge or as a replacement of human thought. The key in its use then is to tune it with human insight to correct mistakes and add surgically to the structure of the AI knowledge base. Perhaps the best method of augmentation is through counterfactuals.
Counterfactuals have the benefit of being built over a human-understandable Structured Causal Model (SCM) and so are very pointed, Not only that, the change that might be induced in the AI knowledge base can be very directed, i.e. systematized with little or no confounding.
The automation of increased AI knowledge base efficacy is the challenge of the future of AI. Human insight and direction is the one method for this to be done consistently and reliably.
