Jevax: Jev-driven EMACS
I plan to try running Jev inside EMACS so that it predicts the next command to execute.
The input is the screen state + task, along with the possible next commands. The expected output is the probability of the next command.
Since every action in EMACS is a command, Jev can do anything this way, absolutely anything.
The only problem is that there are an incredible number of commands, and it is unclear how Jev will handle this.
This might be optimized somehow. For example, the commands could be organized into a tree, first selecting a group of commands and then the right command within that group.