Guidance

Strategy

Developing ideas is often the most difficult part of ELK, especially with so many working on the issue. While researching, we recommend a babble and prune approach, where you write down as many ideas as possible, no matter how “dumb” they seem. As you research, try to build a long list – write down modifications you might make, how to apply to the real world, connections, proposal hypotheses, etc. With this list, prune your ideas by cutting out those you think are less valuable, difficult to accomplish, or succumb to counterexamples. Keep this list though, as it’s often a good starting point if you’re stuck!

What should I do if I’m stuck?

Figure out what works best for you! Often, taking 1-2 days completely off of a project (even if it’s hard) and returning with a fresh mind can help reset your mind. Looking at new sources (e.g. old proposals) and asking another person for help are also good strategies to help. If you tried the babble and prune approach above, you can also go back to the ideas you pruned out and build off those. Even if you’re stuck, there’s always a way out!

Writing Guidelines

Below is a set of guidances modified from the Alignment Research Center to keep in mind when formulating ideas. This can be used in two ways: a) as a place to gain starting points to “babble” ideas, and b) as a checklist for your rough draft to check for errors. Some of this guidance might seem confusing – we recommend coming back to this section after your research.

Reward Function

It’s likely going to be much more difficult (though not impossible) to solve ELK with a comprehensive model, and in general, it’s a good idea to include a term in your reward function (i.e. the function that describes how the AI assesses something) that depends on the structure of your model. While we would be most interested in models that penalize “bad” AI behavior in any model, this is difficult to accomplish. If you wish to do so, it might be useful to limit “gaming” the reward function by minimizing the effects of Goodhart’s Law – making it difficult to increase reward value via non-aligned methods. One could do this by incorporating AI into the reward function and/or adding consistency checks, but we expect unique surprises in this category.

Complexity

In general, we recommend not to consider cases where the model becomes more complex or learns something new as it progresses. However, check that your proposal doesn’t incentivize extra complexity or further learning/modification – if it does, then you should address that case. Making a more complex model only complicates the issue, and ELK is already very difficult with a simpler algorithmic model.

Precision

In order to be considered, your proposal must specify a precise way to approach the problem. If the idea is too broad or informal, then it’s difficult to test your approach without wondering if we’re assessing it correctly. However, if you include a novel counterexample that applies to a broad category of ideas, then you may leave the solution fairly broad if your goal is to prove that the entire category does not work.

Translation

If your proposal seeks to directly describe the AI’s neural network rather than using human simulation to determine the validity of an outcome, you should include a specific reason why your model would prefer direct translation. For example, if your model would assess the SmartVault situation as a whole as “good” or “bad” rather than simulating asking a human if it were “good” or “bad,” include why that might be.