Discovery-Tiny-001: A Diffusion Code Model That Is Honest About Memorizing 20 Functions

discovery-tiny-001 is a public Python repository of mine under github.com/yethikrishna. It holds a README, a 71 line inference.py, three training and model files under code/, and a tokenizer. The weights are a release asset, about 370 MB, which I did not download. I read the README and the scripts. I did not run the model.

What it does

The README describes a roughly 92 million parameter masked discrete diffusion model for code, trained from scratch for 20,000 steps on 20 short Python functions: add, sub, mul and so on up to avg and diff. It reports 20 of 20 exact matches and 20 of 20 parsing when each function is regenerated from a 6 token prefix. It also states plainly what the model is not: it does not generalize, does not follow instructions, and its output outside those 20 prefixes is undefined. I think that honesty is the best part of the repo.

Where it goes wrong

Two details in inference.py. First, the completion call is given the number of tokens to produce, and that number is computed as the length of the original function minus the prefix length. So the model is told how long the answer is. For memorized text that is a mild help, but it is not free, and a real completion would not know it. Second, the checkpoint is loaded with torch.load and weights_only set to False, which allows arbitrary pickled objects. That is common in research code, but anyone downloading a 370 MB file from a release page should know it, and I should say it in the README.

Also, the README's score is on the training set by design. It is a memorization check, and it is labeled that way, so I am not calling it misleading. It just cannot say anything about whether diffusion is a good fit for code.

What I would change

My opinion: load with weights_only set to True, or ship safetensors. Add a held-out function, even one, and report the failure honestly next to the 20 of 20. Add a mode that stops on an end token so the length is not supplied. The question the repo is really asking, whether discrete diffusion can write code it has not seen, needs those three things before it has an answer.