QUESTIONS & problems

Questions & Problems?

One of the goals of this initiative will be to identify: 

• Questions about the notion of predictable AI systems, ecosystems and futures. Questions are aimed at clarifying the conceptual space. They can be technical, non-technical or even philosophical. We will start collecting and refining these questions on the two panels of the March2023 kickstart event.

• "Board" Problems that require actual solutions in the direction of making AI more predictable. Problems can be technical or not but sufficiently precise. We will collect these problems, and potential ideas for solving them prior to the March2023 event, and in the highlight invitations and final session of the event (and displayed on boards, hence the term "Board Problems").

In the months after March 2023, we will work on organising these questions and answers, together with the associated problems and the more promising ideas and directions for solving them. 

Examples of Questions:

List of questions as proposed by participants (with 'x' representing support):

1. Should we infer a set of capabilities as a cognitive footprint of system performance? xxxxxxx

2. What are the pros and cons of delegating predictions to independent (ML) assessors based on existing experiments? xxxxxxxx

3. Can we anticipate the capabilities and risks of future AI systems when deployed in a social context? xxxxxxxxxxxxxxxx

4. Is human-like AI more predictable for humans or are human-likeness and predictability orthogonal? xxxxxxxxxxxx

5. Can we determine causation for AI liability as those situations that were predictable? (REPHRASED ALSO AS A PROBLEM, SEE BELOW: What’s the relation between causation, predictability and AI liability?) xxxxxx

6. What kind of harmful capabilities can we anticipate AI systems to have?

7. Is anthropomorphism a significant hindrance to robust AI evaluation?

8. Are AI Safety and AI Evaluation orthogonal research streams, or two sides of the same coin? 

9. Can predictability of AI be increased through dynamic evaluation? x

10. What’s the role of predictable AI for future industries?

11. What kind of qualitative (causal, explanatory) inference makes sense from quantitative performance measurements?

12. What’s the relationship between explainable AI and predictable AI?

13. When would/should an AI system request help from a human expert? How frequently? What types of collaborations can be set up?

14. Can mechanistic interpretability research eventually help us predict which capabilities models will develop next during training?

15. Are current regulatory frameworks adequate in preventing AI from becoming unregulated? If not, what new mechanisms need to be developed?"

16. What are the potential problems of making big models available for everyone?

17. Does the interaction with AI using natural language make it more predictable, or just the other way around?"

18. How neural network (NN) scaling laws compare to other universal scaling laws across complex systems research? (e.g., Geoff West's superlinear scaling laws identified for cities and social phenomena and Melanie Moses' observations of scaling in computation) 

19. In scaling laws, could the scaling exponents increase with further distributed systems (like ant colonies or the human swarm)? 

20. Are there patterns in the ways that AIs fail to generalise?

21. Can we predict anything about the progress of AI from looking at biological evolution of intelligence?

22. How accurate have previous forecasts of AI progress been, and how can we learn from their mistakes?

23. What role predictability of AI plays for human trust/acceptability in/of AI?

Examples of "Board" Problems:

List of board problems as proposed by participants (with 'x' representing support):

1. How can we determine the range of (co-)operating conditions, instructions and tasks where complex AI-human ecosystems will be robustly good? xxx

2. What are the predictive levels of aggregation and cognitive abstractions that lead to better AI control? x

3. How can we find scaling laws in social scenarios full of emergent phenomena or situations where the law of large numbers doesn't hold? xxxx

4. What tools do we have to anticipate failures and risks when instructing AI systems through natural language, other than red-teaming? xxxxxx

5. What are good indicators of predictable performance? How can we robustly assess predictability? AI evaluation/behaviour repositories to test predictability xx

6. How to perform capability inference from system performance? xxxx

7. How to frame and evaluate the predictability of general-purpose AI? x

8. What’s the relation between causation, predictability and AI liability? xxxxxx

9. How can AI systems recognise "bad" or "potentially bad" things and mark them as such?

10. How to effectively implement human control of AI?

11. Possible ways to control technologies leading to an unstable future

12. How to study the trade-offs between AI predictability and AI competency (or human-AI efficiency)? xx

13. How to extrapolate AI capabilities into the future?

14. How to improve the HCI using predictable AI?

15. How to organize risk management for artificial intelligence systems?

16. How to harmonize the answers and decisions/solutions from AI systems with the current ethical (ideological) constraints/restrictions and rules of human societies?

17. Methods for description and detection of unexpected behavior of large language models and mitigating their risks.

18. Issue of differing ideologies regarding AI *predictability* and their potential wider implications for society.

19. How can we anticipate future distributions and degenerations caused by generative AI, in language, images, etc.?

20. Are scaling laws on engineering results such as LMs contaminated by the human/science bias of keeping up with the trend?