A small language model, given a task prefixed with a description of a recurring decision, will often recognize the decision and fail to act on it. This paper draws together a program of studies on that effect and states what drives it. The effect is carried by the content of the description, not by its form: a plain im...
Sophie Neilson· Zenodo (CERN European Organi...· 0 citations
A small language model, given a task prefixed with a description of a recurring decision, will often recognize the decision and fail to act on it. This paper draws together a program of studies on that effect and states what drives it. The effect is carried by the content of the description, not by its form: a plain im...
Sophie Neilson· Zenodo (CERN European Organi...· 0 citations
A behavioral format assay changes the structure of a prompt and reads whether a model’sanswers improve. Verdict accuracy is the usual measure. We show that it is not sufficient,and that a model’s self-reported basis for its answer is not sufficient either. We gave threelocal language models a structured decision rubric...
Sophie Neilson· Zenodo (CERN European Organi...· 0 citations
A behavioral format assay changes the structure of a prompt and reads whether a model’sanswers improve. Verdict accuracy is the usual measure. We show that it is not sufficient,and that a model’s self-reported basis for its answer is not sufficient either. We gave threelocal language models a structured decision rubric...
Sophie Neilson· Zenodo (CERN European Organi...· 0 citations
A procedure can be written in more than one form. We compare two forms of a duty (a gated procedure that governs a class of work) for repairing a confirmed software defect. The annotated form is written by reading a hazard registry and citing each entry it guards against, so the finished procedure depends on that regis...
Sophie Neilson· Zenodo (CERN European Organi...· 0 citations
A procedure can be written in more than one form. We compare two forms of a duty (a gated procedure that governs a class of work) for repairing a confirmed software defect. The annotated form is written by reading a hazard registry and citing each entry it guards against, so the finished procedure depends on that regis...
Sophie Neilson· Zenodo (CERN European Organi...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.