Dimwitted Sage

This site is being rebuilt. Some pages and posts are still in progress.

AI Experiments

Full disclosure, the bulk of content here will be AI generated. I'm sure the initial content will read robotically, but this will also be a part of experimentation.

There are multiple objectives to the experiments. Number one is to gain a greater understanding of datasets used for precision validation. This in turn will also be used to explore LoRA tuning of models, and we'll look at what kind of problems that smaller models are capable of solving.

Ongoing: Paper Radar, new arXiv papers summarized and compared with recent work as they come in.

About the summary check

Paper Radar only shows summaries that pass an automated check: every number and name in a summary has to appear in the paper. Summaries that fail are withheld, and the paper is listed with a link to its abstract instead.

The check has gaps in both directions. It can withhold a summary over a formatting difference: “Precision on Demand” was withheld because “95%” didn’t match the paper’s extracted text. It can also pass a summary that’s wrong, because it only checks that a number or name appears, not how it’s used. That same summary described the method incorrectly, and the check had no way to catch it.

Better ways to validate output like this are one of the goals of this series.

  1. Summarizing New arXiv Papers with a 0.5B Model